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EXECUTIVE SUMMARY 

This Final Report summarizes the results of an evaluation study of selected 
elementary and secondary schools funded by the California Department of Education under 
the Foreign Language Assistance Act to develop model K - 12 programs in the teaching of 
less-commonly taught languages (LCTLs). Fourteen (14) projects at nineteen different 
school sites received a maximum of four years of funding to offer instruction in Japanese, 
Chinese (Mandarin and Cantonese), Korean and Russian. The evaluation of each model 
project was carried out by a team of language evaluators from Stanford University. 

The evaluation design was guided by six instructional and evaluation questions 
regarding the teaching and learning of the target languages. The six questions addressed in 
this study were: (1) How can proficiency in the four language skills be assessed considering 
the diversity of languages, students’ age, and teaching methods?; (2) Do students make 
reasonable progress in acquiring the language within a school year and across levels (e.g., 
Japanese 1 and Japanese 2) of instruction?; (3) How does motivation influence students’ 
language proficiency attainment?; (4) How do parents’ attitudes toward language study and 
their involvement influence students’ language proficiency?, (5) Does students’ ethnic 
heritage language background play a role in language study and proficiency development?, 
and (6) What challenges and difficulties do teachers and administrators experience in teaching 




of the LCTLs. 
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The Evaluation Team developed an evaluation strategy that relied heavily on teacher 
collaboration in working with students to develop language portfolios and in periodic rating 
of student oral proficiency. The evaluation also depended on student cooperation on 
numerous tasks ranging from preparing audio or video tapes for their portfolio to consenting 
to be interviewed in the language that they were learning. Parents also contributed through 
their completion of a survey questionnaire. 

A multi-method approach was adopted for the collection of language proficiency 
information on students. These methods included: a language portfolio, an oral proficiency 
rating scale (Stanford Foreign Language Oral Skills Evaluation Matrix), an oral interview 
(Classroom Oral Competency Interview), and a self-assessment of proficiency in four 
language skills (Sung Language Assessment Questionnaire). In addition, there was a student 
questionnaire which contained questions regarding motivation for studying the language, 
learning strategies, parental involvement in students’ language study, and background 
information on age, gender, and ethnic heritage language. The questionnaire was 
administered to all students beginning with the fourth grade. A questionnaire was also sent 
home to the parents of all students in the various language programs. The parent survey 
asked information on attitudes toward second language learning, involvement in child’s 
language study, and background information on gender, language study, and ethnicity. 

The information that was collected for this evaluation study was extensive. Over 200 
language portfolios were assembled by teachers and students in school year 1994-95, and 
again in 1995-96. Evaluation Team members reviewed these 400 portfolios and prepared 
summaries of the level of language proficiency evident by the work found in the portfolios. 
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In addition, teachers provided Stanford FLOSEM ratings on over 1,200 students during both 
school years. The data base included two years of FLOSEM language ratings on 569 
continuing students, representing all of the language programs. Another 708 high school 
students provided self-ratings of their language proficiency on two separate instruments (the 
Stanford FLOSEM and the Sung Language Assessment Questionnaire). Members of the 
Evaluation Team administered the Classroom Oral Competency Interview (COCI) to 375 
students. Over 1,200 students from the 4th through the 12th grade completed the student 
survey questionnaire. To complement the student information, surveys were completed by 
nearly 1,600 parents with children from kindergarten through the 12th grade. Finally, 
information was gathered from 16 teachers and 8 administrators about challenges to teaching 
LCTLs and strategies for maintaining their program once the FLAAP grant terminated. 

After analyzing all of the available data, the Evaluation Team was impressed with the 
acquisition achieved by students in the LCTLs. Careful scrutiny of all of the portfolios 
available to the Evaluation Team led us to the conclusion that student portfolios which 
included written and oral samples of work in a foreign language class provided powerful 
evidence that students were achieving adequate mastery across the four language skill areas 
within one school year and also across levels of instruction. Other forms of language 
assessment [teacher FLOSEM ratings, student self-ratings on the FLOSEM and the Sung 
Language Assessment Questionnaire, and COCIs] showed that language development in the 
LCTL occurred within a school year and across levels of instruction with students’ attaining 
increasing competence in the language of instruction. The data also showed that students 
from the same ethnic heritage background as the language studied were able to attain higher 
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levels of proficiency than non-ethnic heritage students. 

Findings from the student and parent survey were also very important in enabling the 
Evaluation Team to identify important relationships that influence language learning. For 
example, motivation [instrumental /integrative] to learn a LCTL played an important role in 
students’ acquisition of the language. However, other considerations such as grade level, 
gender, ethnic heritage status, opportunity to use the language outside of the classroom, and 
parental involvement often exerted more influence in language proficiency than motivation 
alone. 

Although we were not surprised as evaluators by the finding that ethnic heritage 
language plays a critical role in the learning of the languages studied, it is very important 
that language educators keep this fact in mind when they plan and implement foreign 
language programs in the LCTLs. For example, ethnic heritage students reported greater 
reliance on outside classroom use of the language which significantly correlated with greater 
oral proficiency. Another important finding was the significance of parent-reported 
involvement in their child’s language study. The different statistical approaches taken in our 
analyses all revealed the important role played by parents when students enroll in a LCTL 
program. The role of parents is even salient when examined from the perspective of ethnic 
heritage parents. Those parents who reported that they were more involved in their 
children’s language study, in turn, had children who attained high oral language proficiency 
ratings from their teachers. 

Turning to the administration of the FLAAP programs, there was considerable 
variation in how they were managed at each site. However, all teachers and administrators 
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felt that support from the FLAAP grant had been essential in the development and/or 
strengthening of their program. Teachers and administrators all acknowledged numerous 
challenges in the implementation of their program. The challenges ranged from an absence 
of curriculum and materials to certificated teachers. Teachers and administrators stated that 
without the grant it would have been very difficult to overcome some of the challenges posed 
in teaching a LCTL at their schools. Teachers and administrators both believed that students 
were interested in the program at their school, and that to a lessor extent the program was 
supported by parents. However, teachers perceived moderate to little support for their 
program from their principal, school superintendent, and school board. However, this did 
not deter teachers from their efforts to have a strong language program. Finally, teachers and 
administrators indicated that their programs would continue in some way even without grant 
funds, but that it would be difficult to sustain the same type of activity without the support 
that the grant provided. However, it is important to mention in conclusion that nearly all 
teachers expected that they would continue teaching a LCTL even without FLAAP grant 
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Chapter 1 
Introduction 

In recent years there has been considerable interest in the teaching and learning of 
certain foreign languages that are not the traditional offerings in the curriculum of most 
public schools. These languages have come to be called the "less-commonly taught 
languages" (LCTLs) and generally include many of the world languages such as Japanese, 
Chinese, Russian, Arabic, Korean, etc. The importance of Asian languages, in particular, 
has been recognized in California by language educators, parents and students. According to 
the California Basic Education Data System (1995), the number of students enrolled in 
Japanese language classes has risen from 556 in 1982-83 to 5,439 in the 1994-95 school year 
(an increase of 861%). While not as dramatic, enrollment in Chinese classes has increased 
from 1,085 to 2,708 (149.6% increase) during the same time period. The increase in 
enrollment in the LCTLs is most notable in high school foreign language classes, although 
there are some elementary school programs that teach a LCTL. 

Policy makers at both the state and federal level have recognized the importance of 
instruction in the LCTLs. Thus, efforts have been instituted to stimulate elementary and 
secondary schools to offer instruction in the LCTLs. This has occurred through the use of 
federal grants to school districts to initiate programs for students in the LCTLs. For 
instance, one such effort to stimulate demonstration projects in the LCTLs came about 
through the 1991 Foreign Language Assistance Act which provided federal funds to selected 
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school districts who submitted proposals to teach one or more of the LCTLs. In California, 
grants for these model projects in the LCTLs were administered through the California 
Department of Education. These grant-awarded projects were called the Foreign Language 
Assistance Act Program (FLAAP). An important component of the model projects that were 
funded by the California Department of Education was evaluation. The intent of the 
evaluation was to independently determine whether the funded projects met their objectives 
of teaching students the four language skills (listening, speaking, reading and writing) in the 
target language as well as cultural knowledge. The California Foreign Language Project 
(CFLP) at Stanford University was designated as the evaluator for the FLAAP model 
demonstration projects in the LCTLs. 

This Final Report summarizes the results of the evaluation study of selected 
elementary and secondary schools funded by the California Department of Education under 
the aegis of the Foreign Language Assistance Act of 1991 to teach the less-commonly taught 
languages of Japanese, Chinese (Mandarin and Cantonese), Korean and Russian. 

Specifically, 14 projects (6 Japanese, 4 Russian, 3 Chinese, and 1 Korean projects) at 
nineteen different school sites received four years of funding (1992-96) to develop and 
implement a demonstration language program in the teaching of one or more less-commonly 
taught languages. The programs were highly diverse in terms of teacher composition, 
student enrollment and strategies for implementing instruction in the LCTLs. Programs 
were situated in five elementary schools, one middle school, and thirteen high schools in 
Northern and Southern California. 



All funded programs had submitted proposals specifying how they intended to 
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implement LCTL instruction in their school. In the proposal, school officials indicated the 
objectives that they hoped to achieve through the program. The Stanford-based evaluators, 
which consisted of the authors of this report and native speakers of the languages studied, 
henceforth to be referred to as the Evaluation Team, began their study by familiarizing 
themselves with the proposal submitted by each funded program. 

Objectives of the Evaluation 

The evaluation of each model program was carried out by a team of language 
evaluators from Stanford University headed by the authors of this report. The Evaluation 
Team began the process of establishing the evaluation design shortly after the schools 
received their funding from the California Department of Education. The design was guided 
by several broad instructional and evaluation questions which pertained to the teaching and 
learning of the target language and culture. The questions and the research strategies for 

gathering evidence to answer these questions were developed by the Stanford Evaluation 

■) 

Team in collaboration with the teachers and school administrators responsible for 
implementing the model language programs. These questions guided the Evaluation Team 
throughout the four years of funding. Because the programs were all different in grade level 
of students, languages taught, curriculum and program implementation, no attempt was ever 
made to directly compare the programs in the evaluation study. Thus, analyses were carried 
out by comparing languages taught (e.g., Japanese vs. Chinese), levels of instruction (first 
vs. second year), and other variables (e.g., gender and ethnic background of students), but at 
no time were specific programs compared. 
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Instructional and Evaluation Questions 
Based on the objectives for funding the LCTL programs, each project’s goals 
specified in the proposals and knowledge from the literature in foreign language education, 
six instructional and evaluation questions were developed to guide the assessment of the 
model programs in the LCTLs. The questions are as follows: 

1) How can proficiency in the four language skills (speaking, listening, reading, and 
writing) be assessed considering the diversity of languages, students’ age, and 
teaching methods? 

2) Do students enrolled in the less-commonly taught languages (LCTLs) make reasonable 
progress in acquiring the language within a school year and across levels (e.g., 
Japanese 1 and Japanese 2) of instruction? 

3) How does motivation to study the target language differ depending on the language 
program, school level, gender and ethnic background? Does motivation influence 
students’ language proficiency attainment, especially oral proficiency development? 

4) Do parents’ attitudes toward foreign language learning and involvement in their 
child’s language study differ depending on the language program, school level, 
gender, and their ethnic background? Do parents’ attitudes and involvement influence 
students’ language proficiency attainment? 

5) What role does the student’s ethnic heritage language background play in language 
study and proficiency development in the less-commonly taught languages? 

6) What challenges and difficulties do teachers encounter in teaching of these 
less-commonly taught languages? 
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In order to conduct the evaluation of the LCTLs using the guiding questions, it was 
necessary to establish a process that included the development of several new instruments and 
procedures for gathering information. The most important steps taken by the Evaluation 
Team to address each of the six instructional and evaluation questions are summarized in the 
next section of this Report. 

Methodology for Addressing Each Question in the Evaluation Design 
Because the nature of instruction was so important to the success of this evaluation, 
the methods used to assess learning of the LCTLs involved the teachers as much as possible. 
The hope was that by focusing closely on the instructional process with teachers, we could 
use portions of the evaluation plan and outcomes to foster professional understanding of 
effective methods for teaching the LCTLs. We also sought to incorporate their assistance in 
the data collection effort and to make them a part of the Evaluation Team through individual 
and group feedback sessions at the school sites or through group workshops. These features 
are included in the description of the process that the Evaluation Team used in addressing 
each the six questions. 

Question #1: How can proficiency in the four language skills (speaking, listening, 
reading and writing) be assessed considering the diversity of languages, students’ 
age, and teaching methods? 



To address this question, the Evaluation Team first devised a system of student 



FLAAP HNAL REPORT 



16 



portfolios as a form of performance assessment and implemented this in every language 
program. Student portfolios include samples of students’ work that show their performance 
on the four language skills (listening, speaking, reading and writing) as well as the learning 
of cultural information about the people who speak the target language. 

The Evaluation Team also developed an oral proficiency rating scale, the Stanford 
Foreign Language Oral Skills Evaluation Matrix (FLOSEM), that could easily be used by 
teachers to assess students’ oral proficiency development. Teachers were asked to rate their 
students’ proficiency three times (September, January and May) on the FLOSEM during the 
school year. Target language oral proficiency assessment was also carried out on selected 
case study students by means of the Classroom Oral Competency Interview (COCI) at the 
end of the 1994-95 and 1995-96 school years. This assessment was conducted by native 
speakers of the target language who were part of the Evaluation Team. The COCI was 
developed by a committee under the auspices of the California Foreign Language Project 
(CFLP) Policy Board. 

A final instrument that was used to assess writing proficiency was the Classroom 
Writing Competency Assessment (CWCA) which is under final development by the 
California Foreign Language Project. Participating teachers were given training on the 
CWCA and asked to use the protocol to obtain writing samples from their students for 
inclusion in the 1995-96 school year portfolio. 

Finally, we also asked high school students during school year 1995-96 to provide the 
Evaluation Team with self-assessed oral proficiency ratings in the four language skill areas of 
listening, speaking, reading and writing. These data were collected by means of the Sung 
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Language Assessment Questionnaire which is a 40-item scale. Students were also asked to 
self-rate their proficiency using a slightly modified FLOSEM scale. This was done to 
correlate students’ self-ratings on the FLOSEM with their teachers’ ratings. 

Question #2: Do students enrolled in the less-conunonly taught languages 
(LCTLs) make reasonable progress in acquiring the language within a school year 
and across levels (e.g., Japanese 1 and Japanese 2) of instruction? 

In order to answer this question, the Evaluation Team met frequently with teachers, 
made classroom observations, and developed the assessment protocol described above. We 
began by exploring ways to develop student portfolios during the 1993-94 school year. Then 
portfolios were fully implemented during the 1994-95 and 1995-96 school years. As part of 
the portfolio, teachers were asked to use the FLOSEM rating form at least twice a year (late 
September and late May) and preferably three times a year (end of January). FLOSEM 
scores were retrieved by the Evaluation Team and recorded for all students in all language 
programs by all language levels. This procedure allowed us to examine language growth 
within an academic year (September to May). Further, by collecting FLOSEM ratings 
across school years for students who continued from one level to the next in a language 
program, we were able to examine language development longitudinally for some students as 
they moved across two levels of foreign language instruction. A similar procedure was used 
with a smaller number of case study students for whom information on the COCI was 
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The value of the design used in this evaluation study was that it was both longitudinal 
and cross-sectional. The evaluation design was longitudinal because some information (such 
as student portfolios for selected case study students and every student’s oral proficiency 
ratings) was available across a two-year span of time. When viewed from a cross-sectional 
perspective, on the other hand, we can examine changes in language growth by comparing 
different cohorts of elementary students from kindergarten to fifth grade and from high 
school students enrolled in level 1 (introductory) to level 4 or 5 (advanced) classes. 

Question #3: How does motivation differ depending on the language program, 
school level, gender and ethnic background? Does motivation to study the target 
language influence students’ language proficiency attainment, especially oral 
proficiency development? 

To better understand why students enroll in the less-commonly taught language 
classes, the Evaluation Team developed a survey questionnaire which was administered 
twice, once in 1993 and again in 1995. The questionnaire contained two different scales 
which asked students various questions regarding their reasons and interests in studying the 
less-commonly taught language. One scale consisted of items taken from the now classic 
foreign language motivation questionnaire developed by Richard Gardner and colleagues 
(Gardner, 1985). Gardner believes that there are two types of motivation that explain second 
language learning. These are instrumental and integrative motivation for studying a foreign 
language. Numerous studies by Gardner and others have confirmed the existence of these 
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two forms of motivation for learning a new language. The Evaluation Team believed that in 
the absence of much information about why students enroll in the LCTLs, it was important to 
gather information about students’ motivation for language study. The second scale consisted 
of items developed specifically for this evaluation project. Items on this scale asked students 
to indicate their reasons for enrolling in a LCTL and were designed to access information 
about ethnic heritage considerations (e.g., want to speak to a relative), school based factors 
(e.g., friends are taking the class) or other personal reasons (e.g., want to study abroad) for 
wanting to learn the language. 

The survey questionnaire was administered to all students from fourth through the 
twelfth grade. It was felt that students below the fourth grade were too young to fully 
comprehend the questions on the survey so that students from Kindergarten to the third grade 
were excluded from the survey. In order to determine whether a relationship existed 
between motivation and language attainment, students’ scores on the motivational scales were 
correlated with teacher FLOS EM ratings. 

Question #4: Do parents’ attitudes toward foreign language learning and 
involvement in their own child’s language study differ depending on the language 
program, school level, gender, and ethnic background? Do parents’ attitudes 
and involvement influence students’ language proflciency attainment? 

An important dimension seldom examined in second language acquisition research is 
the role of parents in the learning of the language by their children. In this evaluation study. 
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a questionnaire was prepared for administration to parents that asked questions about their 
own language learning experiences and attitudes. In addition, questions were also asked 
about parents’ level of involvement in their child’s language study. In essence, these 
questions sought information on parents’ active involvement as language facilitator for their 
children. To determine whether parents’ reports of involvement matched their children’s 
perception of parental involvement, the same set of questions were also included in the 
student questionnaire. The questions were slightly re-worded to assess students’ perception 
of parental involvement. 

The parental questionnaire was sent home with all students from kindergarten to 
twelfth grade with the request that parents complete the survey and return it to school. The 
questionnaire was prepared in English, Japanese, Chinese, Korean, Russian, and Spanish. 
The questionnaire was administered in the Autumn of 1993. Since new cohorts of students 
entered the various language programs at the beginning of academic year 1994 and again in 
1995, the survey questionnaire was administered to all new students and parents in Autumn 
of 1995 following the same procedures used during the first data collection effort in 1993. 

In order to examine whether parents’ attitudes and involvement influence their child’s 
proficiency attainment, parents’ responses were correlated with students’ oral proficiency 
ratings. 



Question #5: What role does students’ ethnic heritage language background play 
in language study and proficiency development in the less-conunonly taught 



languages? 
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Given the changing demographic profile of students in California with the rapidly 
increasing number of Asian heritage students, this question sought to better understand how 
heritage language background might influence students’ attitudes and motivation to enroll and 
learn one of the four target languages funded by the California Department of Education. 
Thus heritage language became a critical part of the analyses to be reported on in this Final 
Report. In some language programs, there were many same ethnic heritage language 
students and the Evaluation Team felt that it was important to understand whether heritage 
language learners differed from other learners in the programs we evaluated. We were able 
to collect self-designated ethnic heritage information from students and parents from the 
survey questionnaire which asked questions about ethnic and language background. By 
combining heritage language information and motivational data and analyzing to see what 
impact these factors have on students’ learning as determined by such measures as ratings on 
the FLOSEM, we are in a good position to understand how personal factors interact in 
second language instruction. This is especially important given the scarcity of information 
regarding student characteristics and the LCTLs. 

Question #6: What challenges and difficulties do teachers encounter in teaching 
of these less-conunonly taught languages? 

To answer the last question regarding the challenges and difficulties that teachers in 
the less-commonly taught language programs encounter, the Evaluation Team conduced 
interviews with classroom foreign language teachers, foreign language department chair 
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persons, principals, and any other foreign language program-related personnel. At the last 
meeting with teachers and administrators, a survey was also conducted to evaluate the impact 
and accomplishment of the FLAAP grant for their foreign language programs. 

The above six questions serve as a structure to organize the major findings that 
emerge from this evaluation study. The organization of this report will now be discussed. 

Organization of the Final Report 

The instructional and evaluation questions discussed above were used to organize the 
remainder of this Report. Specifically, in Chapter 2 we describe the process engaged in by 
the Evaluation Team to implement the evaluation plan over the four years covered by this 
FLAAP Evaluation Project. Because a review of the literature revealed very little 
information about the teaching and learning of the LCTLs, it was necessary to develop 
and/or use several new instruments to collect data on target language attainment. In other 
studies of the learning of foreign languages, it is known that such factors as motivation to 
learn the target language, learning strategies and parent involvement all appear important in 
learning a second language. Thus an instrument to assess these personal factors was also 
created especially for this evaluation study. Chapter 2 provides information about the 
measurement devices that were developed for use by the Evaluation Team. Chapters 3 
through 6 present the major findings from the evaluation which address the issues of 
students’ proficiency assessment and factors influencing target language proficiency 
development. The chapters which summarize our findings are organized around the six (6) 
questions addressed at the start of this Report. Chapters 3 and 4 describe how we sought to 
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assess target language proficiency attainment as brought up in Questions and #2 and what 
we learned from this effort. Chapter 5 presents information about the students’ reasons for 
enrollment in the LCTLs as mentioned in Question #3. Also included in this chapter is a 
summary of parents’ attitudes toward foreign language learning and their involvement in the 
language study of their child (Question #A). The heritage language background issue in 
relation to the students’ language study including students’ motivation, learning strategies, 
parental involvement and oral proficiency attainment (Question #5) was discussed throughout 
Chapters 3, 4, and 5. Chapter 6 summarizes our findings with teachers and administrators 
about their perceptions of the program. This chapter also describes insights on the 
challenges and difficulties that teachers and students encounter in the teaching and learning of 
the less-commonly taught language programs (Question #6). 
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Chapter 2 

Description of Evaluation Activities 
Description of Project Sites 

When the FLAAP grants were first awarded in 1993, 13 project sites received 
funding. These projects were located in 12 participating schools in both Northern and 
Southern California. Projects were guaranteed two-year grant support and later the grants 
were extended for an additional two years. Thus projects were awarded four years of total 
support. In the 1994-95 school year, one more project was added with four participating 
schools. This project received two years of funding extending through Spring 1996. Also a 
school was added to an existing project site during the final project year. In addition, one 
Russian project changed over the funding period with one school dropping out of the project 
and new schools added. These new projects and schools were all incorporated into the 
evaluation study along with the originally funded projects. However, actual participation in 
the evaluation effort varied widely from project to project. Information on each project site 
(participating schools, language taught and length of participation with the grant) is shown in 
Table 1. 

In the first year of the evaluation study. Evaluation Team members visited each 
project site and had numerous phone conversations with teachers whenever necessary. The 
evaluators discussed with teachers and school administrators the procedures to be used in 
evaluating each program. The evaluation questions which were listed at the opening of this 



[Table 1] FLAAP Grant Participating Projects 



Project 


Participating School 


Language Taught 


City Located 


Length 


(1) 


Clarendon Elementary 


Japanese 


San Francisco 


4 years 


(2) 


West Portal Elementary 


Cantonese 


San Francisco 


4 years 




Hoover Middle 


Cantonese 


San Francisco 


4 years 


(3) 


Arts Magnet Elementary 


Russian 


Oakland 


4 years 




Garfield Elementary 


Russian 


Oakland 


1 year 


(4) 


El Marino Language 


Japanese 


Culver City 


4 years 


(5) 


Saratoga High 


Japanese 


Saratoga 


4 years 


(6) 


Brea-Olinda High 


Japanese 


Brea 


4 years 


(7) 


Sunny Hills High 


Korean 


Fullerton 


4 years 


(8) 


Venice High 


Japanese 


Venice 


4 years 


(9) 


Venice High 


Mandarin 


Venice 


4 years 


(10) 


Venice High 


Russian 


Venice 


4 years 


(11) 


Hilltop High 


Mandarin 


Chula Vista 


4 years 


(12) 


Hilltop High 


Russian 


Chula Vista 


4 years 


(13) 


School of the Arts 


Russian 


San Francisco 


1 year 




Burton High 


Russian 


San Francisco 


3 years 




Lowel High 


Russian 


San Francisco 


- 




George Washington 


Russian 


San Francisco 


- 


(14) 


Mira Loma High 


Japanese 


Sacramento 


2 years 




Merced High 


Japanese 


Merced 


2 years 




Golden Valley High 


Japanese 


Merced 


2 years 




Atwater High 


Japanese 


Atwater 


2 years 
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Report served as the focal questions in organizing the evaluation. It is also important to 
mention in this chronology that the original 13 projects had all been funded prior to the 
awarding of the evaluation grant to Stanford. Thus details for evaluating programs were 
finalized with the site coordinators and teachers after instruction in the less-commonly taught 
languages had already begun. 

The first site visits took placed during the months of May and June 1993. These 
visits served the purpose of acquainting Evaluation Team members with the program and 
staff members at each school site. During these visits, members of the Evaluation Team 
observed and videotaped class teaching for purposes of documenting instruction. In addition, 
interviews were carried out with teaching staff, school administrators, students, and their 
parents to collect baseline data for the longitudinal evaluation. At the same time, the 
Evaluation Team also collected information on existing techniques and instruments for 
assessing oral proficiency in the target languages (Japanese, Chinese, Korean, and Russian). 

During the Fall of the 1993-94 school year, the Evaluation Team re-visited all of the 
project sites and had discussions with coordinators and teaching staff about the evaluation 
plan. This visit focused on working with teachers to develop student language portfolios to 
document language growth. Specifically, the discussions centered on how portfolios could be 
used to show language growth and what types of language-related information needed to be 
gathered for student portfolios. In addition, details were worked out with teachers to 
distribute and collect a questionnaire intended for students and another for their parents. The 
student and parent survey was deemed essential for obtaining information known to be 
related to second language study and attainment, so the importance of the survey was 
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emphasized with teachers who were asked to work with their students to ensure a good 
response return from students and parents. This was especially true for parents of secondary 
students who, it was believed, might be especially difficult to reach by means of a take-home 
questionnaire. 

During the third and fourth year, members of the Evaluation Team continued the 
practice of visiting each site twice a year: Once at the beginning of the school year to report 
the previous year’s evaluation results and discuss the coming year’s evaluation plan, and 
again at the end of the school year to collect student proficiency data by means of the COCI. 

Other than regular school site visits and on-going phone conversations with teachers, 
the Evaluation Team planned and carried out annual meetings with teachers and 
administrators from all participating schools. During these meetings, the evaluation team 
provided an opportunity to have a group workshop on various professional development 
topics (e.g., portfolio assessment, COCI and CWCA training, FLOSEM training, etc.) and to 
discuss general concerns and problems encountered by the teachers of less-commonly taught 
languages. These workshops and discussions were very important because most of the 
teachers who participated in the project had few opportunities for professional development. 

Professional Development Workshops for Teachers 
PortfoUo Assessment Workshop 

Since student portfolios were to be used to document students’ language growth within 
a school year (and also across years of instruction), a series of workshops on portfolio 
development was held for teachers and administrators. At the beginning of the 1993-94 
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school year, members of the Evaluation Team visited each school site and held an individual 
training session with language teachers about the basic information on student language 
portfolios. Teachers and Evaluation Team members discussed the purposes and audiences of 
language portfolios, kinds of artifacts which could be included in a portfolio to show the 
students’ language learning progress, and other logistics such as where to store artifacts. 

After this training meeting, each teacher was asked to think about their respective school plan 
for the student portfolios. A follow-up discussion was held with teaching staff at each school 
site via phone conversations. To strengthen and broaden the idea of portfolio assessment, the 
Evaluation Team with the collaboration of the California Foreign Language Project held a 
workshop on portfolio assessment on November 1, 1994. Every school representative 
brought a sample portfolio with them and shared it with other teachers. Teachers also 
exchanged experiences and problems in developing student portfolios and using them as a 
form of authentic assessment. The invited speaker, Susanne Charlton from UC Irvine, led a 
workshop on portfolio assessment. The speaker focused her comments on strengthening 
teachers’ understanding of portfolio assessment and identified “best” practices to support this 
form of assessment in foreign language education. Emphasis was also given by the 
Evaluation Team to teachers to follow the guidelines and a table of content format so that 
there would be consistency of portfolios across projects regardless of language type or 
grade/level of instruction. 

The Evaluation Team also delivered a mid-point report and explained the evaluation 
procedures for the current year. During this meeting, teachers were encourage to raise any 
questions or concerns related to their foreign language teaching. All questions were 
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answered by either a member of the Evaluation Team or by Dr. Duarte Silva, Executive 
Director of the California Foreign Language Project. At this meeting, all schools were 
informed that the FLAAP would continue to fund them through the 1995-96 school year, 
thus, what we were really doing at the Workshop was planning for an evaluation process for 
the next two years. 

COCI Training Workshop 

In addition to workshops on portfolio development and use, an intensive two-day 
workshop for LCTL teaching staff was held at Stanford University in February 1995 on the 
use of the Classroom Oral Competency Interview (COCI). Since it was planned to collect 
students’ oral proficiency data by means of the COCI as part of the evaluation plan, it was 
decided prior to such data collection that it would be helpful to teachers to offer a 
professional development workshop on the administration and scoring of the COCI. Through 
the COCI workshop, all teachers would be informed on how the Evaluation Team planned to 
conduct oral proficiency assessments with the identified case study students at all school 
sites. Further, it was hoped that if teachers received training on the COCI, they might 
subsequently use the COCI in their own assessment of students’ proficiency. 

Training on the COCI was conducted by Silvia Jones who was on the committee that 
developed the COCI and who is one of the major trainers for teachers on the COCI. In the 
workshop she was assisted by our language evaluators, who had previously been trained 
themselves on the COCI. During the first day, each teacher and language coordinator 
received COCI training including a demonstration interview with ESL students. ESL 
students were interviewed to ensure a common language among all the language teachers. 
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On the second day, teachers practiced administering the COCI to learners of their target 
language (e.g., Japanese, Russian). The COCI training with teachers was carried out with a 
two-fold purpose. First, the Evaluation Team wanted to let each teacher know what kind of 
oral proficiency interview instrument the language evaluator would use with the designated 
case study students at their school during late Spring. The other purpose was to encourage 
the classroom foreign language teachers to give serious consideration to using the COCI in 
their oral proficiency assessment of students when called upon to do so. 

Portfolio Development 

In the field of foreign language education, the use of student portfolios as a means of 
documenting and charting student growth in the four language skills (listening, speaking, 
reading, and writing) as well as cultural knowledge was very much at a developmental stage 
when the Evaluation Team began to work with project sites. The Evaluation Team viewed 
working with the project sites as an essential part of its efforts to contribute to the practice- 
side of foreign language education and to finding useful methods of performance-based forms 
of assessment. The use of portfolios was one such method of assessment explored as part of 
the evaluation plan. 

We began with the idea of student portfolios by reviewing the literature on portfolio 
assessment. A search of the relevant literature revealed that most of the published material 
on portfolios was in the area of science and language arts education. No literature was 
located that dealt with the use of portfolios in the foreign language classroom. Using the 
existing literature, we developed a plan for portfolios that incorporated ideas about criteria 
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for inclusion of materials in a portfolio, frequency of information collection, and 
documentation of materials selected for the file. Once this plan was developed, we discussed 
portfolios with teachers through an individual training session in the 1993-94 school year. 

The content of the training focussed on discussions with teachers about the purposes and 
audiences of portfolios in their classes, how to assemble the portfolio, what kind of artifacts 
would be included and how often, etc. In thinking about how best to implement a plan for 
portfolios, we decided to give teachers the freedom to choose their audience and purpose of 
the portfolio. We believed that if the use of portfolios was to be implemented meaningfully 
by teachers, it was important to maintain flexibility in how teachers defined the audience and 
purpose of their portfolios, since our schools varied between elementary and secondary levels 
and included four different languages. After presenting the options for portfolios to teachers 
at the various sites, we simply indicated that there would be Guidelines to follow to make 
portfolios as uniform as possible across sites and to maximize their utility as alternative 
forms of assessment. 

The Evaluation Team asked each school site to select six students from each level 
(e.g., Japanese 1, Japanese 2, etc.) or grade (Kindergarten, first grade, etc.) in each 
language program to serve as case study students for the portfolio assessment portion of this 
project. After reviewing portfolios at the end of the second year of the Project, the 
Evaluation Team refined their Guidelines and devised a Table of Contents for student 
portfolios. Based on the Guidelines and Table of Contents, schools continued to assemble 
student portfolios during the 1994-95 year of the project evaluation. The use of portfolios 
was well received by participating teachers and also by students in the program. The 



FLAAP FINAL REPORT 



31 



portfolio guidelines were again revised after review and study of portfolios collected at the 
end of the third year. The most recent draft of the revised portfolio guidelines is included as 
Appendix 1. 

Based on our experience in guiding language teachers through the process of 
developing and implementing student portfolios in a foreign language classroom, the 
Evaluation Team has given presentations on using portfolios in foreign language education at 
several professional meetings. For example, members of the Evaluation Team made a 
presentation at a round table discussion on portfolios in the classroom at the 1995 Annual 
Meetings of the American Educational Research Association (AERA) in San Francisco. 
Subsequently, the paper on which the presentation was based was submitted for publication to 
the Foreign Language Annals which is the official publication of the American Council of the 
Teaching of Foreign Languages (ACTFL). The paper was published in the Fall 1996 issue 
of the Foreign Language Annals . A copy is included as Appendix 2. In addition, 
presentations on implementation of student portfolios in the foreign language classroom were 
also given at the Annual Meetings of the American Council on the Teaching of Foreign 
Languages (ACTFL) in November 1995, Los Angeles and again in November 1996, 
Philadelphia. 



The Stanford FLOSEM 

The Stanford FLOSEM was first piloted with selected students at every school site at 
the end of the 1993-94 school year. Based on teacher feedback and reflection on the rating 
matrix by the Evaluation Team, the matrix was subsequently revised and put into use during 
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school years 1994-95 and 1995-96. The Evaluation Team met with teachers at all the school 
sites and discussed the FLOSEM matrix and how the instrument should be used in rating 
students oral proficiency in the target language. Instructions for the FLOSEM along with the 
scoring matrix are attached to this report as Appendix 3. 

Student Self-ratings of Target Language Proficiency 
At the end of the 1995-96 school year, high school students were asked to self-rate 
their language proficiency level. The self-rated proficiency task consisted of two parts: the 
Stanford FLOSEM and the Sung Language Assessment Questionnaire. The FLOSEM matrix 
was modified slightly, including some re-wording to facilitate interpretation by students of 
some descriptions in the matrix. The Sung Language Self-Assessment Questionnaire 
consisting of forty questions, ten questions for each of the four language skill areas 
(listening, speaking, reading and writing), was also used to gather information about 
students’ proficiency attainment in the language they were studying. Items on the 
questionnaire were answerable using a five-point Likert scale depending on the degree of 
comfort students had in using the target language in a given situation. This questionnaire 
was developed and used successfully in an earlier study by Sung (1995) with adult 
participants. The value of this questionnaire is that it provides self-assessments in the four 
language skills. A copy of the self-proficiency questionnaire for students is attached as 
Appendix 4. 
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Student Interviews 

During the month of May or June in 1995, all school sites were visited for purposes 
of conducting oral proficiency examinations with the designated case study students at each 
grade or language instruction level. The California Oral Competency Interview (COCI) was 
administered by native speakers of each language for assessing the level of oral proficiency 
of the case study students. The administration of the COCI was video and/or audio-taped so 
that the examiners’ could review the tape to ensure that they score the students’ interview 
properly. Teachers were provided with copies of the COCI videotapes of their students at 
the beginning of Fall semester. The results of the COCI proficiency assessments were also 
shared with all teachers. 



Student and Parent Questionnaire Survey 
In order to examine students’ and parents’ attitudes and motivation of learning these 
less-commonly taught languages (LCTLs) as well as parental involvement in their child’s 
language study, the Evaluation Team conducted a questionnaire survey of students from the 
4th grade through to high school seniors enrolled in a language class. The questionnaire for 
students consisted of four parts. Part 1 consisted of 8 items comprising a scale of 
instrumental and integrative motivation for students enrolled in a LCTL program (e.g., 
"studying Japanese is important because it will be useful in getting a job in the future"; "it 
will allow me to meet with a variety of people"). This scale was adapted from a similar 
scale used previously by Gardner (1985). Part 2 was designed specifically for this study and 
consisted of 14 items which tapped other motives or reasons for studying these specific 
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languages. These motives included: 1) motivation based on ethnic heritage (e.g., "this is my 
heritage language"); 2) school-related motivation (e.g., "I heard that the Chinese teacher at 
our school was good"); and 3) reasons based on personal interests (e.g., "I thought that 
studying Korean would be more interesting than studying European languages such as 
French, Spanish, or German"). Part 3 of the questionnaire contained a series of items that 
asked students about various strategies that they may or may not have used in their language 
study. There is growing evidence that successful language learners use various strategies to 
acquire language skills in a second language. Items for this section of the questionnaire were 
adapted from Oxford (1992) who has provided extensive information about language learning 
strategies known to be effective in language classrooms. Part 4 of the questionnaire asked 
students to indicate the level of parental involvement in their foreign language study. 
Examples of the 9 items include the following (e.g., "my parents encourage me to practice 
the foreign language as much as possible"; "my parents show considerable interest in my 
foreign language study"). 

The questionnaire for parents was divided into two parts. Part 1 consisted of 9 
questions and asked parents about their attitudes toward foreign language learning in general 
(e.g., "studying a foreign language is an enjoyable experience"; "I wish to learn many 
foreign languages"). The second part which also consisted of 9 items asked parents to 
indicate their level of involvement in their child’s language study (e.g., "I encourage my 
child to practice the foreign language as much as possible"). These last 9 items of the 
parental questionnaire were identical to the Part 4 of the student questionnaire. Items on 
both the student and parent questionnaires were arranged on a 7-point Likert scale from 
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"strongly agree (6)" to "strongly disagree (0)". 

The student and parent questionnaires are included in as an Appendix 5 of this report. 
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Chapter 3 

Portfolios as Assessment Devices 
Student Portfolios 

The evaluation design called for teachers to identify six (6) case study students from 
each language level (or grade) for the purpose of tracking their progress throughout their 
participation in the program. Because we were specifically interested in formal instruction 
and learning of the LCTL by students who had no familiarity with the language prior to 
instruction, we asked each participating teacher to try to select case study students who did 
not belong to the same ethnic/language heritage background as the language they were 
learning in school. Beginning in the 1993-94 school year, all teachers worked with their six 
case study students to prepare portfolios. Since this was the first time that teachers had used 
portfolios in their language classes and because there were no available guidelines, this was a 
year of trial-and-error for all in learning how to implement portfolios. Portfolios were 
submitted by teachers to the Evaluation Team in Summer 1994. These portfolios were 
studied and comments prepared for teachers regarding: (a) the contents of the portfolios and 
(b) suggestions for improving the portfolios. Based on this trial year with portfolios, the 
Evaluation Team refined the guidelines for classroom use of portfolios and worked with 
teachers to improve how the overall quality of students’ portfolios. 

Thus in academic year 1994-95, teachers were again asked to work with case study 
students to develop portfolios that could be used to document language growth in the LCTL. 
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The same request was made in 1995-96. Portfolios were received between mid-June and 
mid-July in both 1995 and 1996. Approximately 200 portfolios were submitted for our 
evaluation during 1995 and again in 1996. 

After the 1994-95 portfolios were received, they were submitted to a content analysis 
by the same native speaking evaluators who had visited each program in their assigned 
language and carried out the COCI testing with these case study students. Thus, the 
evaluators were familiar with the teachers and students whose portfolios they content- 
analyzed. The objective of the content analyses of the portfolios was to gather evidence that 
students had demonstrated growth in the four language skill areas from September to May. 

The evaluators prepared an individualized report of each students’ growth in the target 
language based on materials in the portfolio. These individualized reports for each of the 
case study students became a permanent record in our data system. At the same time, our 
portfolio reports were also given to the teachers in Autumn 1995 for their feedback and 
discussion regarding student progress in the LCTL. The intent of providing teachers with the 
reports was to encourage teachers to use the portfolios to determine whether they needed to 
adjust their curriculum to enhance the target language acquisition of students. 

Because there is no commonly agreed upon practice for conducting a content analysis 
of portfolios, the language evaluators met frequently to discuss similarities and differences 
noted in the case study portfolios. Out of these discussions, a format was decided upon for 
the content analysis and reports to be prepared by the evaluators. The intent of the analysis 
was to create a record of the specific contents of the portfolio and to categorize the contents 
by language skill (listening, speaking, reading, writing and cultural knowledge) 
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demonstrated. In addition to this, evaluators provided their impressions of the student’s 
growth in the target language based on their global interpretation of the materials in the 
portfolios including FLOSEM and COCI ratings. 

Analysis of the 1994-95 school year portfolios showed that teachers who had prior 
experience in using portfolios as a means of performance assessment of language 
development provided much better documentation of student language learning. For 
instance, teachers who participated in our pilot phase of portfolio development (school year 
1993-94) and who followed our guidelines for developing language portfolios organized the 
items in the portfolio more systematically and provided sufficient materials for evaluators to 
adequately document the student’s language learning progress. Also portfolios which 
contained a table of contents proved to be very useful for both teachers and students as they 
planned and implemented portfolios in the foreign language classroom. 

In 1995-96, the portfolios of case study students submitted by teachers to the 
Evaluation Team were divided into two groups for analysis. In the first category were case 
study students for whom we had amassed at least two years of language data using our 
system of student portfolios. The second category consisted of case study students for whom 
we had only a single year (1994-95 or 1995-96) of data because students: (a) graduated from 
high school in 1995, (b) discontinued their language study after June 1995, or (c) served as 
new and/or replacement cases in 1995-96 for students who had left the language program 
through either (a) or (b) above. There were a total of 71 students for whom we had two 
years of language portfolio data available. The language evaluators compared the growth in 
language ability across the two years using all information available in the portfolios which 
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included writing samples, quizzes, video and/or audio tapes of oral presentations, cultural 
projects, 2 or 3 FLOSEM ratings and COCI ratings done by our language evaluators. 

Since the accumulation of summaries based on the content analyses of portfolios 
amounts to approximately 1,200 pages of text, we will only provide a sampling of the 
portfolio summaries prepared by the language evaluators to give the reader an idea of the 
process engaged in to make the portfolios meaningful in this evaluation study. What follows 
are the portfolio summaries for five prototypical students enrolled in a high school Japanese 
or Korean language program across a two-year span of time. In order to ensure 
confidentiality of students, they are identified by their first name only. Students names, 
language studied and level of instruction are as follows: 



Name 


Laneuaee and Level 


John 


Japanese 


I ton 


Yvonne 


Japanese 


n to m 


Mark 


Japanese 


nito IV 


Wmiain 


Korean 


I ton 


Cesar Korean III to 


IV 



Portfolio summaries as prepared by the language evaluators for each of the five 
students across the two-year span of language study are presented in their entirety here. 

Study of these summaries reveal important insights into both the strengths and weaknesses of 
the portfolios for accessing information about a student’s growth in the target language. By 
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taking the cross-sectional approach used here to examine portfolio summaries from Level I 
through Level IV instruction, the reader of this Report will have a good understanding of the 
materials found in a portfolio and how such materials were used by the language evaluators 
to infer linguistic and communicative competence of the students in the four language skill 
areas. 

It is also important to observe the differences in portfolio summaries between the 
1994-95 and 1995-96 periods. Teachers and students were adept in the development of 
portfolios in the second time period and as a consequence the language evaluators’ reports 
were more extensive for the second period. Also, in their second period reports the language 
evaluators commented when appropriate on a case study student’s language growth between 
the two time periods when such evidence was available. 
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Summaries of Three Students Enrolled in High School Japanese Program 



Name: John 
Level: 1 

Date of Evaluation: July 1995 
Comments: 



Writing: 



All writing samples and tests were reviews of material covered in class. Since many activities 
were completed throughout this year, his overall writing improved gradually. 



Reading: 

to read. 



His reading skill was good for a first level student. On the 6/1 test, there was a long passage 
He answered all the questions. 



Listening: In the 10/28 audio tape, he had a difficult time understanding the teacher’s language. 

Speaking: Very good pronunciation in audio tape for 10/28 teacher-student interview. To check his oral 

skill progress throughout the year, it would have been better if the second teacher-student 
interview would have been included in the portfolio. 



COCI Rating: Formulaic High 



* The COCI was administered on 5/25. Student was able to engage in this activity, even though, I was a 
stranger to him. His listening and speaking skills are good for his level. 

FLOSEM Score: 10/25/94: Total score = 6 (C-1, F-1, V-1, P-2, G-1) 

2/1/95: Total score = 7 (C-1, F-1, V-1, P-3, G-1); 

7/15/95: Total score = 9 (C-1. 5, F-1. 5, V-1, P-3, G-2) 
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Name: John 
Level: 2 

Date of Evaluation: July 17, 1996 

(1) Types of Artifacts: 

Table of contents, student’s notes, kanji practice, dialogue, daily schedule, translation test, dictionary form 
practice, kanji test, fall semester final exam, sentence practice sheets, quiz, homework: questions & answers, an 
essay for CWCA, spring final project, audio and video tapes 

(2) Quality of Student* s Work: The worksheets and practice sheets were very neat with good handwriting. 
However, his test scores were not always good. His fall semester final exam score was 69% and quiz in March 
for the negative form showed that he had only 8 correct answers out of 22 questions. 

(3) Organization of Portfolio: 

Student’s portfolio was well prepared. It was ordered chronologically and it had his own table of contents and 
his comment about why he chose the tasks and what he learned from the them. 

(4) Student *s Language Growth. 

Writing: The CWCA was rated in the Formulaic Mid range. Unfortunately, we did not have a CWCA writing 
sample for the beginning of the year. So it was not possible to compare growth in structure of sentences, 
phrases, transitions and vocabulary from the beginning to the end of the school year. Also, no information was 
available on how long it took to finish the essay and if the student used a Japanese dictionary or received outside 
help. From other worksheets or tests/quizzes, he seemed to have some trouble in grammar, particularly in the 
use of particles. 

Reading: His reading skill was adequate for the level of instruction, but he was not sure of some words. 

Speaking/Listening: In the interview with the Japanese teacher in spring, he could not answer many of the 
questions she asked. This was a surprise to me since he could understand my questions on the COCI in May. I 
thought that his language skills had constantly improved over the year. 

Comparison of this year and last yearns portfolios: 

Last year’s portfolio was also well prepared and it indicated that his listening comprehension was not good. This 
year’s portfolio also showed that his listening comprehension has not improved much. However, based on my 
impression of the 1996 COCI, I believe that the student has improved gready in listening comprehension. This 
is very interesting because my impressions seem to be supported by the teacher’s FLOSEM ratings, which show 
the student having improved in oral skills by 2.5 points over the year. Although the improvement is not large, it 
does show that his oral abilities in Japanese have improved. The teacher’s ratings on the FLOSEM corroborate 
my COCI rating (Formulaic High in 1995 and Created Low in 1996). As for writing, there is still much room 
for improvement. 

1995 COCI Formulaic High 

1996 COCI Created Low 



1994- 95 FLOSEM 

1995- 96 FLOSEM 

1994- 95 Grades 

1995- 96 Grades 




7, 9 

13.5, 14.5 



B + 
B 
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Name: Yvonne 
Level: 2 

Date of Evaluation: July 1995 
Comments: 



Writing: 


Excellent writing all the time with very few mistakes. In order to see her progress in writing, 
the passages/ essays would have been a good tool. Unfortunately, no passages/essays in free 
writing were available in the portfolio. 


Reading: 


Excellent. Many reading materials were in the test, such as translations from Japanese to 
English, reading Japanese questions, fill-in-the-blank questions etc. 


Listen: 


Very good listening comprehension 


Speaking: 


Obviously, she learned from mistakes. She made a mistake on denying a compliment in a test. 
But she used the correct form for this structure when it was called for in the COCI on 5/25. 
The audio tapes (1/10, 4,24) showed very clear and excellent pronunciation. Improvement in 
her speaking skill was amazing. 


Culture: 


Throughout the year, she learned a lot of culture and culture related vocabulary. The test of 
culture included in the portfolio was very good. 


COCI Rating: 


Created Low 



FLOSEM Scores: 10/25/94: Total score 12 (C-2, F-2, V-2, P-4, G-2); 

2/1/95: Total score = 12 (C-2, F-2, V-2, P-4, G-2); 
7/15/95: Total score == 13 (C-2, F-2, V-2, P-4, G-3) 



* I feel that her comprehension ability is higher than the 2 shown on the FLOSEM rating. 
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Name: Yvonne 
Level: 3 

Date of evaluation: July 19, 1996 

(1) Types of Artifacts: 

Table of contents, student’s reflective notes, translation dialogue, verb quizzes, homework, self-introduction, 
grammar quiz, fall semester final quiz, 10 sentences, dialogue, vocabulary quiz, dialogue, test, dialogue, 
CWCA essay, spring semester final exam (90 %) 

(2) Quality of Student *s Work: 

Very well done 

(3) Organization of Portfolio: She had many dialogue practices or translation activities and also verb tests. 
Artifacts were almost in chronological order with student’s reflective notes. 

(4) Student *s Language Growth: 

Writing: Her writing samples didn’t have many mistakes. For self-introduction artifact, there were rough and 
final drafts, but her rough draft was very good with only a few teacher corrections. She seemed to be very 
skilled in wnting even at the beginning of the year. Her CWCA essay was well done, so it was scored in the 
Created Low range. 

Reading: Fluent. Clear pronunciation, though sometimes foreign accent was noticed slightly. 

Speaking/listening: It seemed to be good. If the video or audio tape had provided impromptu talk with the 
teacher or a native speaker, it would have been helpful to see her speaking or listening skill more clearly. 

Comparison with this year and last yearns portfolios: 

Last years’ portfolio showed her excellent understanding in Japanese. This year, the portfolio also provided 
evidence of her good command of Japanese. Her improvement in the language seemed to be steady over the 
year. This impression is also supported by the teacher. 



1995 COCI 


Created Low 








1996 COCI 


Created Low 








1994-95 


FLOSEM 


12, 


12, 


13 


1995-96 


FLOSEM 


13, 


15.5, 


16.5 


1994-95 


Grades 


A-I-, 




A-l- 


1995-96 


Grades 


A, 




A 
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Name: Mark 

Level: 3 

Date of Evaluation: July 1995 

Comments: 

Write: His passage was very good. He was able to make his own sentences. During this year, more 

complicated phrases and useful expressions were taught in class. The questions about shapes and 
directions were very complicated and difficult to express in writing. However, he did a very good job 
on these questions. 

Read: There were many opportunities in the portfolio to see the students ability to read across situations 

including reading comprehension questions, vocabulary quizzes, Kanji translations, etc. Since his 
performance was almost always perfect on tests, I assume that his reading must have been good. 

Listen: Most tests had listening comprehension questions and he was able to answer many of them. 

Speak: He appeared nervous during the video taping, but his pronunciation was very clear. 

Culture: He learned a lot of cultural information through out the year as evidenced in materials in the 

portfolio. 

Vocabulary: His Kanji writing of new vocabulary words was very beautiful. 

COCI Rating: Formulaic High 

FLOSEM Score: 10/25/94: Total score = 15 (C-3, F-3, V-2, P--4, G-3 ); 

2/1/95: Total score = 15 (C-3, F-3, V-2, P-4, G-3) 

7/15/95: Total score = 15 (C-3, F-3, V-2, P-4, G-3) 
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Name: Mark 
Level: 4 

Date of evaluation: July 16, 1996 

(1) Types of Artifacts: 

Table of contents, student’s notes, kanji quiz, worksheets, an essay "Why Japanese is fun" in Nov., fall final 
exam (86%), spring final project (dialogue), two essays written in class for CWCA (within 15 min.). 

(2) Quality of Student *s Work: 

As long as writing was focussed, the quality of his ability to produce creative sentences was very high. 

(3) Organization of Portfolio: 

During the first semester, he prepared all the material for the portfolio including his table of contents and self- 
evaluation. However, during the second semester the organization and work in the portfolio decreased in 
quality. The student didn’t do what he was supposed to do because of his philosophy or what he claimed was a 
widely approved custom at the school which according to an essay in the portfolio indicated that seniors didn’t 
have to do homework. The teacher did supplement his portfolio with materials including two of his in-class 
essays for the CWCA which were written on the same day, June 12th. 

(4) Student *s Language Growth: 

Writing: I believe he has great potential in writing in Japanese. One CWCA written sample (topic #\) was to 
take only 5 to 10 minutes while the other CWCA (topic #2) took 15 min. to write. Both essays were written in 
class, according to the teacher’s note. There were some English words remaining in his Japanese sentences 
because he didn’t use a dictionary or seek assistance from the teacher. His writing showed that he express 
himself with a range of sentence types in Japanese. I rated his CWCA essays as Created Low, but near Created 
Mid. Overall, he showed improvement in writing was throughout the school year. 

Reading: His reading skill was okay, but with some foreign accent. 

Speaking /listening: Both speaking and listening skills were very good. 

Comparison of this year and last yearns portfolios: 

Last year’s portfolio showed that the student’s writing skill was already high, but I still see even more 
improvement in this year’s writing samples. The student growth in the oral skills (listening and speaking) 
appeared to be much slower than his development of writing. The teacher’s ratings on the FLOSEM support my 
impressions of the student from the COCI and content analysis of the portfolios. 



1995 COCI 

1996 COCI 

1994- 95 FLOSEM 

1995- 96 FLOSEM 

1994- 95 Grades 

1995- 96 Grades 



Formulaic High 
Formulaic High 
15, 15, 15 

13, 16.5, 16.5 
A, A 



A-, A- 
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Summaries of Two Students Enrolled in High School Korean Program 

Name: William 
Level: Korean 1 
Date of Evaluation: July 1995 

I. Audio 

Student was able to ask and answer the questions without any difficulty. There was a slight tint of a 
foreign accent in his speech, but it did not disturb the oral communication. His fluency was nearly free 
from any pauses or hesitations in this activity. 

II. Video 

A. Show and Tell (May 17, 1995) 

William talked about his tenth birthday party. He memorized his oral presentation and was able to 
present it with good fluency and pronunciation. His presentation consisted of a series of short sentences 
with the occasional use of conjunctions. 

B. Impromptu Talk ( June 12, 1995) 

William was able to answer most of the teacher’s spontaneous questions without any extra help. His 
ability to speak in Korean reflected his performance in the COCI interview. 

III. Portfolio 

A. Vocabulary/ Spelling/ Grammar Exercises 

1 . Practice writing Korean characters. This particular activity required practice in combining the 
consonants and vowels to create syllables. He seems to have grasped the concept involved in combining 
the vowels and consonants to form syllables. However, at this stage the strokes in his handwriting seem 
slightly unnatural. However, throughout the year, his handwriting did improve. 

2. Learning log. There are vocabulary, grammar and spelling exercises included in the log. William 
seems to be putting a lot of effort into his studies. His log is neat and there are hardly any repeated 
spelling or grammar mistakes. The two logs also show progress in William’s writing, reading and 
listening abilities. 

3. Honorific. William also had practice with the informal and formal. He seems to have a firm grasp of 
the uses of the informal and formal forms. Also, his sentences were consistently correct in their verb 
tenses. 



B. Tests 

William’s test performance was outstanding from the beginning to the end of the year. This obviously reflects 
the amount of effort he is putting into his studies. 

C. Listening/Speaking Activities 

1. Dictation/Review Questions. William’s first attempt at dictation considering that it was towards the 
beginning of the year was quite good. Although he made spelling errors, he seems to have gotten the 
gist of combining the consonants and vowels to form the syllables. In his second attempt at the end of 
the year, his ability to write down what he heard was near flawless. Also, William was able to read 
and answer questions in his own words without any difficulty. There are hardly any spelling or 
grammar errors and the sentences were written in the correct verb tense. 
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2. Cooking. William’s group discussed the procedures and ingredients involved in making kimbab. 
Despite a few spelling mistakes, William was able to write down the elements his group had discussed. 

3. Show and Tell Peer evaluation. William was able to write some complete sentences describing the 
emotional experiences of his classmates. For some cases he was only able to get down a word of two. 
There were some spelling and grammar mistakes as well. 

D. Creative Writing/Class project 

William’s writing ability has improved with great success throughout the year. From being able to write only a 
series of short formulaic sentences, towards the end of the year his sentences consisted of some conjunctions 
and prepositional phrases. For example, "I looked into the mirror and then brushed my hair". His writing style 
read more smoothly. There were hardly any spelling, grammar or verb tense mistakes. William’s strongest point 
in writing is that he is able to tie the sentences together in the paragraph with a connecting thought or idea. 

Overall Comment: 

There was great progress in William’s oral and written proficiency. His written proficiency seems to be a little 
better than his oral proficiency. His works show that he has put much effort into his studies. He also has a large 
vocabulary. William shows great potential to be a successful language learner. 

William’s COCI rating was formulaic low. He was able to understand some of the questions and was limited in 
being able to answer the questions. However, there seems to be quite a difference in his performance between 
practiced oral presentations and spontaneous oral activities in that his oral proficiency in the practiced 
presentations seemed to be much better than his proficiency in the spontaneous activities. Nevertheless, 
considering that it is his first year, his oral proficiency is appropriate. 

William received a 6 at the beginning of the year for his Flosem rating. (1 in everything except pronunciation in 
which he received a 2). At the end of the year he received all (2.5 comprehension, 2.5 pronunciation, 2 
vocabulary, 2 fluency, 2 grammar). It is with total agreement that William has shown much development within 
the past academic school year. 
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Name: William 
Level: Korean II 
Date: August 25^ 1996 



I. Types of Artifacts 

Videotaped oral presentation (group puppet show - "The Ant and the Pigeon"); creative writing samples (e.g., 
paragraphs, sunmiaries, dialogues); comprehension exercises (e.g., listening and reading); grammar/ vocabulary 
exercises (e.g., verb tenses); table of contents 

II. Quality of Work 

The organization of the portfolio is high-average. There were a variety of artifacts and in the table of contents 
the student specified what skills the activity targeted. There were also reflective notes posted on every artifact 
which showed what the student learned from the activity. In comparison to last year’s portfolio, there was a 
decrease in the number and variety of materials included in the portfolio. 

This student’s writing ability is in the formulaic mid range. Although there are many grammatical errors in 
particle usage and verb endings as well as spelling errors, William seems to have made progress in being able to 
use transitions and varied sentences structures in his writing. The majority of his sentence structures are 
simple, but towards the end of the year the paragraphs are showing evidence of complex sentence structures. It 
is evident from his writing samples that there is a growth in his writing styles. He is able to describe events in 
chronological order as well as sununarize concisely and also describe processes. 

III. Language Growth 

As was the case last year, William seems to have made more progress in his writing skills than in his oral 
skills. There was no development in William’s COCI performance. He received a formulaic low both years. 
Although he has good pronunciation and seems to have fair listening comprehension skills, he had difficulty 
formulating expressions orally. His writing skills seem to have improved from last year. Although he still 
needs to work on his spelling and grammar, he is starting to break apart formulaic expressions and is not as 
dependent on formulaic sentence patterns. 

The FLOSEM ratings indicated that William has improved his oral proficiency in both years. William had low 
oral proficiency at the beginning of this school year; however, he has made rapid progress in his oral 
proficiency. His ratings increased a whole level at the end of the year. However, the FLOSEM ratings indicate 
that William is at a much higher oral proficiency level than what the COCI rating shows. In sum, William 
seems to have put a lot of effort into his studies. He received an A for both semesters last year as well as the 
1995-96 Fall semester. His writing skills have also improved over the past two years and according to his 
FLOSEM ratings, his oral proficiency seems to have improved as well. However, he may need to work on 
being able to speak more spontaneously in impromptu situations in Korean. 
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Name: Cesar 

Level: Korean III 

Date of Evaluation: July 1995 

I. Video 

His speech seems to be marked by a mixture of a foreign accent as well as an accent from a Korean dialect 
which he has probably picked up at home. He needs to improve both his fluency and pronunciation. His 
presentation on his life consisted of sentences in the creative and formulaic range connected by conjunctions. 
There were grammatical errors but he was able to narrate in a chronological manner. 

II. Portfolio 

A. Writing/reading 

(story summary, book report, information report, creative writing) 

The majority of Cesar’s portfolio consisted of samples of his writings. Many of his sentences are formulaic, but 
he is making many attempts to recombine formulaic expressions which result in new creative sentences. The 
artifacts show that he is able to comprehend and summarize a story into his own words, report on a factual 
event, write a book report as well as write an essay about his life just to name a few of his acquired skills. As 
a result of his increasing competency to write, his writings are getting longer and more interesting to read 
because of his use of various sentence structures. It looks as if he should improve his handwriting, spelling and 
grammar as well as increase his vocabulary. 

B. Listening/speaking (listening comprehension) 

He included one sample which shows his ability to listen to a story that was read to him and rewrite the gist of 
it in his own words. Judging from his rough draft, he needs to work on his grammar and spelling 

C. Tests 

His performance on tests ranged from grades A to C. Most of his errors were grammatical. 

Overall comments: 

Cesar’s oral and written proficiency seem to be fair in comparison to the other students of his group. He needs 
to work on his pronunciation and perhaps his fluency when speaking. Also, he needs to work on his spelling 
and grammar in his writing. Despite these areas that need improvement, it is evident through the artifacts that 
he has made some progress in his writing ability for he is starting to use a variety of sentence structures. 

His COCI rating was created mid. His ability to comprehend seemed to be better than his ability to produce 
utterances. However, despite the use of some literal translations from English in his Korean speech as well as 
some grammatical errors, he was able to understand and provide the interview with the appropriate responses. 

In Cesar’s case I do not see that much of a gap between his spontaneous oral performance and his practiced one 
as was seen in the video tape. 

At the beginning of the year the teacher gave him a FLOSEM rating of 14 (2 fluency, 3 vocabulary, 3 
comprehension, 3 grammar, 3 pronunciation). However, at the end of the year he received a 16 (3 .5 
comprehension, 3 fluency 3 vocabulary, 3 pronunciation, 3.5 grammar). This seems to be a pretty accurate 
description of Cesar’s progress. 
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Name: Cesar 
Level: Korean IV 
Date: August 25, 1996 

I. Types of Artifacts 

Table of contents; videotaped oral presentation; writing activities (e.g., story creating, essays, book report); 
reading comprehension and listening comprehension activities ( e.g., questions and answers) 

II. Quality of Work 

The organization of this student's portfolio is average. There is some indication of language growth in his 
writing samples, but in general the portfolio lacks a diversity of artifacts that show the student's progress in the 
various skills of language learning. However, based on the various writing assignments, it is clearly evident 
that Cesar’s writing ability is in the created high range. Although he tends to write very colloquially, there is 
some evidence that ideas are beginning to flow across paragraphs. He seems to be able to use various sentence 
constructions, but he still needs to work on his grammar in particular verb forms and spelling. There is also 
some evidence that he translates Korean into English in order to comprehend its meaning. That is, in many of 
the question and answer activities, Cesar had translated the Korean sentences into English which was not a 
requirement of the activity. Cesar also needs to try to improve his penmanship. It was often difficult to 
decipher the syllables of the Korean words. 

III. Language Growth 

Although Cesar tends to write as if he were speaking, his writing has improved a great deal in that he is able to 
convey meaning clearly through written form. There is also some evidence of planned paragraphs emerging 
from Cesar’s writing. His COCI rating for both years is created mid. He still needs to work on his fluency in 
expressing his thoughts more clearly and completely. However, his pronunciation had improved tremendously 
in that his dialectal accent that was so prominent in his speech last year had diminished this year. 

Cesar’s FLOSEM ratings show that his oral proficiency had greatly improved over the past two years. 

Although Cesar has shown some improvement in his oral proficiency, the FLOSEM ratings seem to be an 
over-assessment of his present oral ability. The final FLOSEM rating indicates that he is at a near-native like 
stage in his oral proficiency. In my opinion, his oral proficiency is not quite at that level. He still needs to 
work on expressing his thoughts and ideas in oral paragraphs. Cesar received an A for the 1994-95 Fall 
semester, and A- for the Spring semester. During the 1995-96 Fall term Cesar received a grade of A, grades 
were not submitted for the Spring semester for 1995-96. 
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In addition to the five (5) selected portfolio reports presented here, language 
evaluators also were instructed to prepare summary tables of the more quantitative 
information contained in the portfolios. In order to give the reader a perspective on another 
aspect of the material in a student’s portfolio, Tables A - E present information on students 
from five different high school programs for whom we had two years of language 
information available in a portfolio. The tables present FLOSEM, COCI, and CWCA 
ratings, plus a global rating for the quality of the portfolio itself, and finally letter grade in 
the class if provided by the teacher. These tables present information on a total of 72 case 
study students who continued the language program for two years. The tables are arranged 
with 1994-95 student information on the left hand side and 1995-96 information on the right 
hand side. Each row in a table constitutes information for one student across the two year 
span of time. Finally, student information is arranged by instructional level such that Level 
1 students appear in the top rows, followed by Level 2 students, etc. Remember that when 
students are designated as Level 1 on the left hand side of the table, they are Level 2 
students on the right hand side of the table. This gives us a two-year picture of the 72 
students for whom information is summarized on these tables. 

One entry found in the tables merits additional comment. In the tables is a column 
marked "Portfolio Global". This indicates our effort to develop a rubric by which we could 
evaluate the organization and contents of a portfolio. As can be seen in the tables, most of 
the ratings in this column are "Excellent" (or rating of 3) or "Average" (rating of 2). 
According to our established rubric, a portfolio judged to be "Excellent" includes the 
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Table A: 1994-96 Evaluation Summary Table for Continuing Students 
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Table B: 1994-96 Evaluation Summary Table for Continuing Students 
School Name: HIJKLMN High School Language Taught: Japanese 
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following: a table of contents, varied language samples, audio/video tape, student’s reflective 
notes, and clear evidence of language growth over the academic year. An "Average" 
portfolio was marked with by less varied language samples to indicate growth across the four 
skill areas, entry of materials at irregular intervals, but still sufficient evidence of language 
growth across the academic year. Finally, a "Poor" portfolio, of which there were some, 
was marked by few language samples of any type and which were added to the portfolio in 
such a way as to not make the progression of second language development evident. 

Summary 

Our experience indicates that the practice of using portfolios in the foreign language 
classroom was worth the time and effort that teachers and students gave to developing 
portfolios. When the goal of a portfolio is to show language growth across an academic year 
and a plan is followed to enter materials in the portfolio that show such growth, then the 
practice of portfolio use as a form of performance assessment has been achieved. Our 
experience also shows that teachers and students need training in portfolio development. 
Portfolios as a form of assessment differ from simple files in which to place a student’s 
work. We believe that our evaluation of case study student portfolios provides powerful 
evidence that these selected non-native speakers of the target language were achieving 
adequate mastery across the four language skill areas within one school year and also across 



the levels of instruction. 
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Chapter 4 

Oral Proficiency Assessment Results 

The Evaluation Team collected students’ oral proficiency data throughout the 
evaluation process. Students’ oral proficiency in the target language was examined by three 
different means of assessment. These oral proficiency measures are: teacher FLOS EM 
ratings, student language self-ratings, and COCIs completed with the case study students. 
First, classroom language teachers were asked to assess every student’s oral proficiency by 
means of the Stanford FLOS EM three times a year during the 1994-95 and 1995-96 school 
years. Second, every high school student was asked to self-rate their own oral proficiency 
level using the Stanford FLOSEM at the end of the 1995-96 school year and also asked to 
complete another self-rating proficiency scale for four language skills developed by the 
Evaluation Team (Sung Language Assessment Questionnaire). Third, language evaluators 
visited each school site in Spring and conducted face-to-face interviews with our designated 
case study students using the COCI. 

The results of the oral proficiency assessment will be described to show how much 
progress students made during one school year and also across the various language levels of 
instruction. Growth in language proficiency will first be described by each of the assessment 
instruments and the correlation between the different types of assessment instruments will 



then be examined. 
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Stanford FLOSEM 

Growth in Target Language Proficiency Within the School Year 

Classroom language teachers were asked to rate their students’ oral proficiency using 
the FLOSEM in September, January, and May. Since some teachers were unable to 
complete the mid-year rating, only the September and May FLOSEM ratings for the 1995-96 
school year were compared. Proficiency scores were collected from one thousand three 
hundred nineteen (1,319) students at the beginning of the school year and from one thousand 
one hundred eighty-nine (1,189) students at the end of the school year. Comparisons of the 
two FLOSEM scores were computed by the paired t-test procedure and the results showed 
that students made significant progress in their oral language proficiency within one school 
year, t (1,131) = 24.747,/? < .0001. Significant improvement in oral proficiency ratings 
were found for both the elementary school students, [t (369) = 7.237, p < .0001], and high 
school students, [t (740) = 27.035, p < .0001]. Detailed analyses of the teacher FLOSEM 
ratings between the beginning and end of the school year by each language program showed 
that students in most of the language programs made significant gains in oral proficiency (p 
< .05 level) within a school year, except for the elementary and middle school Cantonese 
programs. Mean FLOSEM scores collected at the beginning and end of the school year with 
t-test results for significant differences are shown in Table 2. 

In Table 2, students’ proficiency growth in each level of instruction was also clearly 
noticed. Students in levels from I through IV in most language program showed a significant 
increase [minimal p < .05] in oral proficiency from September to May. Significant growth 
in oral proficiency was not observed, however, in high school Russian levels I and III and in 
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[Table 2] Students’ FLOSEM Oral Proficiency Progress 
in the 1995-96 School Year 

(Number of students recorded in parentheses; NGM: no level marked; NS: not significant) 



school level 


language 


beginning rating 


final rating 


test of significance 


Elementary 


Japanese 


13.477 (373) 


14.306 (386) 


t (351) = 7.136, p< .0001 




Kinder 


8.778 (63) 


12.426 (94) 


t (61) = 3.653, p<. 001 




1st gr. 


10.772 (79) 


11.409 (77) 


t (75) = 6.044, p < .0001 




2nd gr. 


14.515 (66) 


15.038 (66) 


t (65) = 6.446, p<. 0001 




3rd gr. 


14.952 (62) 


14.745 (47) 


t (46) = -0.125, p = .901, NS 




4th gr. 


16.118 (34) 


16.909 (33) 


t(32) = 4.807,p<.0001 




5th gr. 


17.727 (33) 


18.313 (32) 


t (31) = -0.833, p = .411, NS 




NGM 


16.806 (36) 


17.459 (37) 


t (35) = 4.312, p < .0001 




Cantonese 


19.203 (64) 


21.300 (20) 


t (17) = 1.338, p = .199, NS 


Middle 


Cantonese 


25.000 (23) 


24.130 (23) 


t (20) = -1.325, p = .200 NS 


High 


Japanese 


9.315 (494) 


12.014 (466) 


t (453) = 21.785, p<. 0001 




level I 


5.126 (234) 


8.550 (202) 


t (199) = 15.770, p< .0001 




level II 


11.869 (134) 


13.579 (133) 


t (129) = 18.040, p < .0001 




level in 


13.821 (78) 


15.103 (78) 


t (76) = 9.678, p<. 0001 




level IV 


15.436 (47) 


17.138 (47) 


t (45) = 8.5 18, p<. 0001 




Mandarin 


9.409 (93) 


12.968 (63) 


t (59) = 6.979, p<. 0001 




level I 


7.474 (57) 


10.017 (30) 


t (26) = 2.249, p < .05 




level II 


10.600 (20) 


14.158 (19) 


t(18)= 13.335, p < .0001 




level in 


13.846 (13) 


16.500(11) 


t (10) = 14.750, p < .0001 




level IV 


19.000 (3) 


22.000 (3) 


insufficient data for test 




Korean 


13.584 (166) 


18.179 (151) 


t (147) = 21. 152, p<. 0001 




level I 


1.329 (38) 


7.986 (35) 


t (34) = 23.452, p < .0001 




level II 


8.803 (38) 


13.403 (31) 


t (30) = 16.985, p < .0001 




level in 


14.061 (33) 


18.194(31) 


t (30) = 18.090, p < .0001 




level rv 


20.259 (27) 


25.180 (25) 


t (21) = 14.782, p < .0001 




level V 


28.633 (30) 


29.534 (29) 


t (28) = 3.527, p < .001 




Russian 


8.085 (106) 


9.113 (80) 


t (78) = 2.287, p<. 05 




level I 


5.972 (53) 


6.742 (31) 


t (29) = 2.738, p = .136, NS 




level n 


7.783 (23) 


9.091 (22) 


t (21) = 2.692, p<. 05 




level in 


10.768 (28) 


10.400 (25) 


t (24) = -0.632, p = . 533, NS 




level rv 


30.000 (2) 


30.000 (2) 


insufficient data for test 




n 

s 



0 
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the third (3rd) and fifth (5th) grades of the elementary Japanese programs where teachers 
recorded little or no growth in the target language from September to May. 

Growth in Target Language Across Language Levels 

Students’ FLOSEM scores were also examined by their level of instruction. For high 
school students, FLOSEM scores collected at the end of the 1994-95 and 1995-96 school 
years were analyzed by their level of instruction (e.g., Japanese I and Japanese II). One-way 
Analysis of Variance (ANOVA) results showed that there was a significant difference on the 
FLOSEM scores by instructional level, F (4, 601) = 334.093, p < .0001 for the 1994-95 
school year, and F (4, 535) = 278.199, < .0001 for the 1995-96 school year, indicating 
that there was a significant cross-sectional increase of students’ oral proficiency by language 
level. 

Thus, steady improvement in students’ target language proficiency as assessed by the 
teachers’ FLOSEM ratings across levels of instruction was found as presented in Table 2. 

An important feature of the growth between levels of instruction that requires highlighting is 
noted between language levels III and IV for the high school Mandarin, Korean and Russian 
programs. Although the sample sizes are extremely small for two language programs 
[Mandarin level IV (N=3) and Russian level IV (N=2)], the enrollment in Korean level IV 
was large (N=29) to argue that in these level IV classes the large FLOSEM increase was 
due most likely to the fact that more language heritage students are enrolled in these 
advanced language classes. For example, over 90% of the students in the Mandarin, Korean 
and Russian level IV classes were from heritage language background households. This 
contrasts with the fact that in Japanese level IV classes only about 10% of the students were 
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either Japanese full or mixed heritage students. 

Growth in Target Language Proficiency for Continuing Students 

For students who continued in their language instruction from the 1994-95 to 1995-96 
school year, FLOSEM ratings taken at the end of each school year were compared to 
examine language development across language levels. A paired t-test comparison was 
computed for the 569 continuing students for whom ratings were available for the two-year 
period. The results showed that continuing students regardless of language level made 
significant progress in their oral proficiency attainment between the end of the 1994-95 and 
the 1995-96 school year, t(568) = 16.266, p < .0001. Significant growth in target language 
proficiency was found for both elementary and high school students [t(220) = 2.193, p = 
.029 for elementary students, and t(339) = 25.831,/? < .0001 for high school students]. As 
noted in Table 3, detailed analyses on growth in foreign language oral proficiency for 
continuing students by each language program and instruction level showed that students in 
most of the language programs and levels made significant improvements (p < .05 level) 
across two school years except for the elementary Cantonese program and several of the high 
school levels. We had FLOSEM ratings for only three continuing students in the elementary 
Cantonese program and although their ratings showed an increase across school years, there 
were too few students for statistical purposes. 

In taking a closer look at the Russian program, a group of students from one school 
site repeated the level I class, and the FLOSEM ratings for the two academic years showed 
that they did not significantly improve in oral proficiency across the two-year period. Also, 
students who advanced from Russian level II to level III showed some improvement in their 
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[Table 3] Continuing Students’ FLOSEM Oral Proficiency Progress 
from the 1994-95 to 1995-96 School Year 



school level 


language 


# of stu. 


1994-95 


1995-96 


test of significance 


Elementary 


Japanese 


218 


14.151 


14.667 


t (217) = 2.001, p<. 05 




K-lst 


42 


7.952 


9.357 


t (41) = 3.492, p < .001 




1 st-2nd 


58 


11.603 


13.647 


t (57) = 4.605, p<. 0001 




2nd-3rd 


26 


17.442 


14.731 


t (25) = 2.484, p < .05 




3rd-4th 


27 


18.037 


17.185 


t (26) = -1.172, p = .252, NS 




4th-5th 


28 


17.429 


18.571 


t(27) = 2.102,p<.05 




NGM 


37 


17.554 


17.459 


t (36) = -0.189, p = .851, NS 




Cantonese 


3 


17.000 


21.000 


t(2)= 1.922, p = .195, NS 


Middle 


Cantonese 


7 


25.286 


22.286 


t (6) = -2.696, p < .05 


High 


Japanese 


196 


10.796 


15.099 


t (195) = 32.037, p<. 0001 




Jp MI 


116 


8.741 


13.711 


t (1 15) = 29.755, p<. 0001 




Jp IMII 


48 


13.010 


16.792 


t(47)= 19.316, p < .0001 




Jp m-iv 


29 


15.603 


18.190 


t(28)= 10.578,p < .0001 




Mandarin 


25 


12.200 


15.300 


t (24) = 8.827, p<. 0001 




Mnl-n 


12 


9.250 


13.083 


t (1 1) = 5.877, p<. 0001 




Mn Il-m 


11 


14.182 


16.500 


t(10)= 1 1.292, p < .0001 




Korean 


68 


16.757 


21.250 


t (67) = 9.809, p<. 0001 




KrI-II 


12 


12.625 


12.625 


t (11) = 0.000, p = 1.000, NS 




Kr II-III 


22 


14.318 


17.988 


t (21) = 9.050, p<. 0001 




Kr III-IV 


20 


17.375 


24.375 


t (19) = 7.172, p< .0001 




KrIV-V 


14 


23.475 


29.357 


t (14)= 10.498, p < .0001 




Russian 


52 


7.827 


9.058 


t (51) = 3.581, p<. 001 




Rs I-I 


8 


5.250 


6.000 


t (7) = 0.728, p = .470, NS 




Rs i-n 


20 


6.550 


8.700 


t (19) = 3.755, p<. 001 




Rs ii-in 


24 


9.750 


10.375 


t(23)= 1.457, p = .159, NS 



ERIC 
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FLOSEM mean ratings increasing from 9.75 to 10.38, but the increase was not statistically 
significant. The minimal growth in oral proficiency among students in Russian classes 
contrasted with the significant improvements found with high school students in the other 
language programs. Table 3 shows the mean FLOSEM ratings collected for all students in 
all language programs and levels at the end of the 1994-95 and 1995-96 school years along 
with t-test results for significant differences. 

Comparisons of 1994-95 and 1995-96 Target Language Proficiency Development 

Other than comparing individual student progress within and over the school year by 
means of paired t-tests, the language program’s "Overall Proficiency Level" was also 
examined by comparing this year’s language program proficiency results with last year’s 
student outcomes. Table 4 summarizes each language program’s FLOSEM ratings by the 
various language levels for the 1994-95 and 1995-96 school years. Perusal of Table 4 shows 
large gains in student oral proficiency for the high school Japanese language programs. The 
proficiency changes were especially apparent in Japanese levels II and IV. Another very 
noticeable finding in our data was an enrollment increase, especially in Japanese language 
programs. Although some enrollment increases were observed in the Mandarin and Korean 
programs, the sharpest enrollment increase was found in the Japanese language programs 
which went from 310 students in 1994-95 to 466 students in 1995-96. 

High School Students’ Self-Rated Proficiency 
A total of 708 high school students participated in the self-rating proficiency task — 
407 students in Japanese, 72 in Mandarin Chinese, 148 in Korean, and 81 in Russian. 



[Table 4] Comparison of Overall Oral Proficiency on the FLOSEM 
between 1994-95 and 1995-96 School Years 

(The number of students in the program is recorded in parentheses) 



school level 


language 


1994-95 


1995-96 


test of significant 


Elementary 


Japanese 


14.950 (309) 


14.306 (386) 


t (693) = -0.982, p = .326, NS 




Cantonese 


15.119 (67) 


21.300 (20) 


t (85) = 3.592, p<. 001 


Middle 


Cantonese 


24.875 (8) 


24.130 (23) 


t (29) = -0.409, p = .686, NS 


High 


Japanese 


10.884 (310) 


12.014 (466) 


t (774) = 3.858, p < .0001 




Mandarin 


11.675 (57) 


12.968 (63) 


t(118)= 1.590, p = .114, NS 




Korean 


17.735 (147) 


18.179 (151) 


t (296) = 0.559, p = .577, NS 




Russian 


13.265 (136) 


9.113 (80) 


t (214) = -4.077, p < .0001 




75 
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Students’ self-ratings on the FLOSEM were compared by means of a Pearson Product 
Correlation with their teacher’s FLOSEM ratings scores collected at the same time. When 
students’ ratings across all language programs were compared to those of their teachers’, 
findings revealed that the students’ self-rated FLOSEM scores and their teachers’ FLOSEM 
ratings were significantly correlated, r = .708; p < .0001. Significant correlations between 
students’ and teachers’ ratings were also found for all four language programs: r = .567; p 
< 0001 for Japanese; r = .435; p < .001 for Mandarin; r = .767; p < .0001 for Korean; 
and r = .621; p < .0001 for Russian classes (see Table 5). 

An interesting finding in these rating scores was the fact that students’ self-ratings 
(Mean = 15.121) were consistently higher than the teachers’ ratings (Mean = 13.056) and 
the difference between students’ and teachers’ ratings proved to be statistically significant, t 
(1, 646) = 11.143;/> < .0001. As can be noted in Table 5, the rating differences between 
students and teachers were greater for students in the lower instructional levels (Levels I, II 
and III) but for upper level students there was almost no rating difference between students 
and teachers. A similar difference was found in every language program, except the 
Mandarin Chinese programs, where the teachers’ ratings were higher than students’, not 
significantly though. It is clearly depicted in Figure 1. Table 5 provides detailed 
information of student-teacher rating differences in each language program by levels of 
instruction. 

Self-rated proficiency ratings on the Sung Language Assessment Questionnaire were 
subjected to a confirmatory factor analysis to determine whether the four blocks of items 
[speaking, listening, reading and writing] demonstrated the same statistical consistency with 




[Table 5] High School Students’ Self-Rated FLOSEM Ratings 
Compared to Teachers’ Ratings 

(Number of students recorded in parentheses; NS: not significant) 



language 


correlation 


student rating 


teacher rating 


test of significance 


Japanese 


0.567* 


13.815 (401) 


12.114 (398) 


t (391) = 7.940, p< .0001 


level I 


0.304* 


11.962 (171) 


8.503 (171) 


t (167) = 10.680, p < .0001 


level n 


0.315* 


13.750 (114) 


13.544 (114) 


t (112) = 0.636, p = .526, NS 


level in 


0.402* 


16.294 (68) 


15.609 (69) 


t (67) = 1.417, p = .161, NS 


level rv 


0.742* 


16.923 (39) 


17.436 (39) 


t (38) = -1.247, p = .22, NS 


Mandarin 


0.435* 


12.543 (70) 


13.491 (55) 


t (52) = -0.588, p = .559, NS 


level I 


0.183 


11.667 (36) 


10.250 (22) 


t (20) = 2.357, p = .029 


level n 


0.634* 


13.889 (18) 


14.158 (19) 


t (17) = -0.339, p = .738, NS 


level in 


0.665* 


13.500 (11) 


16.500 (11) 


t (10) = -7.858, p< .0001 


level rv 


- 


17.333 (3) 


22.000 (3) 


t (2) = -2.843, p = .105, NS 


Korean 


0.767* 


21.132 (148) 


17.631 (134) 


t (133) = 7.235, p < .0001 


level I 


0.650* 


15.671 (35) 


7.986 (35) 


t (34) = 9.236, p < .0001 


level n 


0.551* 


19.431 (29) 


13.552 (29) 


t (28) = 7.497, p< .0001 


level in 


0.499 


20.259 (27) 


18.120 (25) 


t (24) = 2.491, p< .05 


level rv 


0.709* 


23.614 (22) 


25.048 (21) 


t (20) = -1.835, p = . 081, NS 


level V 


0.921* 


29.000 (24) 


29.625 (24) 


t (23) = -1.926, p = .067, NS 


Russian 


0.621* 


12.744 (78) 


9.366 (71) 


t (67) = 5.880, p < .0001 


level I 


0.369 


9.814 (35) 


7.071 (28) 


t (25) = 2.027, p < .05 


level n 


0.093 


13.921 (19) 


9.600 (20) 


t (18) = 4.401, p < .0001 


level m 


0.037 


14.818 (22) 


10.238 (21) 


t (20) = 4. 193, p< .0001 


level rv 


- 


30.000 (2) 


30.000 (2) 


insufficient data for test 



[Figure 1] 



High School Students’ Self-Rated Oral Proficiency 
on the Stanford FLOSEM with Teachers’ FLOSEM Ratings 
by Levels of Instruction 



□ Students' Self-Rated FLOSEM 

_ _ _* Teachers' FLOSEM Ratings 




Instructional Levels 
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the high school students as they did with the original sample of adults for whom the scale 
was initially designed. This statistical procedure is a necessary step to ensure the validity of 
this type of questionnaire. Factor analysis confirmed that the 10 items which formed each of 
the four language clusters loaded as predicted and that the measure could be used with the 
high school data. 

The eigenvalues for a one-factor solution for each of the four scales were very high 
(over 5.0) and percent of total variance explained for each scale was: Listening (56.893%), 
Speaking (63.004%), Reading (65.622%), and Writing (69.533%). Mean scores for each 
language skill on the Sung Language Assessment Questionnaire were computed for all 
participating students. Pearson correlations were computed to examine the inter-relationship 
between mean scores on each of the four language skills. Inter-correlations between all four 
language skill areas were significant (p < .0001 level), and this was true for all four 
language programs. When students’ self-rated proficiency ratings were examined by 
instructional level, the inter-correlations among the four language skills were again high and 
significant (p < .001 level) for all instructional levels. 

Students’ self-ratings on the Sung Language Assessment Questionnaire were then 
compared to their self-rated FLOSEM scores. Self-rated FLOSEM scores correlated 
significantly (p < .0001) with all four self-rated scales on the Sung Language Assessment 
Questionnaire. Significant correlations between the two instruments were also obtained when 
self-rating scores were examined for each of the four language programs. In every case, the 
correlations were highly significant (p < .0001). However, students’ self-rated proficiency 
on the four language skills did not always correlate significantly with teachers’ FLOSEM 



FLAAP nNAL REPORT 



61 



ratings. 

Self-rated FLOSEM scores and self-rated proficiency on the Sung scale for the four 
language skills were examined by language program type, level of instruction, and students’ 
ethnic heritage background. First, students’ self-proficiency was significantly different by 
the level of language instruction, F (4, 670) = 106.181, p <. 0001. The higher the 
instructional level, the higher the self-rated FLOSEM scores (see Figure 1). This 
instructional level difference (on the self-rated FLOSEM) was found between every level of 
instruction from the first year through Level 5 of language instruction. The significant 
instructional level difference was also found with self-rated scales on the four language skills 
of the Sung Language Assessment Questionnaire (see Figure 2), F (4, 679) = 69.530, p < 
.0001 for listening comprehension; F (4, 680) = 72.404, < .0001 for speaking; F (4, 

679) = 60.713, < .0001 for reading comprehension; and F (4, 677) = 80.094, < 

.0001 for writing. Instructional level differences on self-rated scores for these four language 
skills were found even when the significance level (p < .001) was set very stringently. The 
only contrast that was not significant was between Level 2 and Level 3. There was no 
significant difference on self-rated proficiency on the Sung scale between the second and 
third language levels and this was true across all four language skills. However, there was a 
significant difference on the self-rated FLOSEM between these two levels of language 
instruction, p = .004, and also on the teachers’ FLOSEM ratings, p < .0001. 

Students’ self-proficiency was significantly different by language program type. The 
significant difference was found with self-rated FLOSEM, F (3, 693) = 92.238, p < .0001, 
and also with the Sung instrument for all four language skills: listening [F (3, 703) = 
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Proficiency Level on the Sung Scale 



[Figure 2] 



High School Students’ Self-Rated Language Proficiency for 4 Skills 
on the Sung Language Assessment Questionnaire 
by Levels of Instruction 



□ Listening 

Speaking 

s Reading 
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94. 178, /j < .0001]; speaking [F (3, 704) = 83.400, < .0001]; reading [F (3, 703) = 
83.001, p < .0001]; and writing [F (3 , 701) = 86.785, < .0001]. These significant 
results were all due to the fact that the proficiency level for students in the Korean program 
was significantly higher than those of the other three language programs (see Figure 3). 

Students’ ethnic heritage was also entered into the analysis of students’ self-rated 
proficiency. There was a significant difference on the students’ self-rated proficiency by 
their ethnic heritage. First, FLOSEM self-rating scores differed by ethnic background, F (2, 
503) = 86.799, p < .0001, with the highest FLOSEM ratings for ethnic heritage students 
(M = 19.876), followed by mixed heritage students (M = 14.208), and with the lowest 
ratings given by non-ethnic students (M = 13.537). The same pattern of significant 
proficiency differences were obtained on the Sung instrument for the four language skills: 
listening [F (2, 510) = 105.730, /? < .0001]; speaking [F (2, 511) = 78.879, /? < .0001]; 
reading [F (2, 510) = 94.729, p < .0001]; and for writing [F (2, 509) = 92.986, p < 
.0001] (see Figure 4). Tukey’s HSD multiple comparisons showed that proficiency of ethnic 
heritage students was significantly higher than that of the other two groups, whose 
proficiency was not significantly different from each other. 

COCI Ratings by Language Evaluators 

Students’ communicative proficiency was also measured by means of the Classroom 
Oral Competency Interview (COCI) during the last several weeks of the 1994-95 and 
1995-96 school years. Since this method of measuring oral proficiency requires a 
face-to-face interview lasting for around 7 minutes, COCI interviews were conducted only 
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[Figure 3] 



High School Students’ Self-Rated Language Proficiency 
by Language Program Type 




Language Program Type 



Proliciency Scores on the Sung FLOSEM Proflciency Scores 



[Figure 4] 



High School Students’ Self-Rated Language Proficiency 
by Ethnic Heritage Background 
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with our six case study students from each language program and language level. The 
Evaluation Team’s language evaluators visited each school site in the later part of May to 
conduct the COCI interviews. COCIs were administered to a total of 375 students over the 
last two years: 199 students in the 1994-95 school year and 245 students in the 1995-96 
school year. Of these, there were 71 continuing case study students for whom we have 
COCIs for both school years. These continuing students were all high school students 
because in the 1994-95 school year, COCI ratings were not assigned to elementary school 
students after the interviews were completed because technically the COCI was designed for 
use with students in high school level 2 or higher foreign language classes. However, in 
1995-96 we decided to use the COCI rating scale even with the elementary level students. 

The COCI scale has ten grades and each grade was assigned a numeric code that 
could then be used in statistical analysis. The numeric scale which was devised for COCI 
ratings extended from 1 for " Prefunctional", 2 to 4 for "Formulaic Low" to "Formulaic 
High", from 5 to 7 for "Created Low" to "Created High", and from 8 to 10 for "Planned 
Low to Planned High. Overall, 1995-96 COCI ratings for case study students (M = 
3.208; n = 245) were slightly lower than the 1994-95 ratings (M = 3.538; n = 199). Even 
after deleting those elementary school students who received low ratings on the COCI and 
who did not have ratings in the 1994-95 school year, the 1995-96 COCI ratings were still 
lower (M — 3.450; n = 189) than the 1994-95 ratings. Detailed COCI scores by each 
language program are shown in Table 6. Ratings for each language program show that 
1995-96 ratings are lower than 1994-95 scores, except for the Mandarin programs where the 
most recent COCI ratings were slightly higher than in the previous school year. The lower 



[Table 6] COCI ratings in 1994-95 and 1995-96 School Years 



(The number of students in the program is recorded in parentheses) 



for Ail Students 



School level 


Language 


COCI in 94-95 


COCI in 95-96 


Elementary 


Japanese 


NA 


2.393 (56) 


High School 


together 


3.538 (199) 


3.450 (189) 




Japanese 


3.440 (100) 


3.222 (90) 




Chinese 


3.111(36) 


3.250 (32) 




Korean 


5.069 (29) 


5.000 (30) 




Russian 


2.971 (34) 


2.919 (37) 



for Continuing Students 



school level 


language 


COCI in 94-95 


COCI in 95-96 


test of significance 


High 


together 


3.479 (71) 


3.972 (71) 


t (70) = 3.753, p < .0001 




Japanese 


3.675 (40) 


3.850 (40) 


t (39)= 1.000, p = .323, NS 




Mandarin 


3.333 (9) 


4.111 (9) 


t(8)= 1.941, p = .088, NS 




Korean 


4.714 (7) 


4.714 (7) 


t (6) = 0.000, p = 1.000, NS 




Russian 


2.467 (15) 


3.867 (15) 


t (14) = 8.573, p<. 0001 



FLAAP FINAL REPORT 



64 



portion of Table 6 also provides information on the continuing 71 case study students’ COCI 
ratings. For all continuing case study students, COCI ratings were significantly higher this 
year than in 1994-95, t (70) = 3.972, p = .001. However, when comparisons are made by 
each language program, the only significant difference found was that between ratings in the 
two time periods for the Russian language students. 

Using the COCI ratings shown in the top portion of Table 6, a one-way analysis of 
variance was computed for language program type for each school year separately. A 
significant difference was found for each school year: F (3, 195) = 10.215, p < .0001 for 
1994-95 COCI ratings and F (3, 241) = 16.051,/? < .0001 for 1995-96 ratings. Multiple 
comparisons of each language program type yielded findings showing that the Korean 
program recorded significantly higher ratings than all other language programs (p < .0001). 
However, no other comparisons were significant. This finding was identical in both time 
periods. The significant difference was due to the fact that case study students in the Korean 
program had significantly higher proficiency ratings on the COCI than students in the other 
language programs. 

Since the COCI was administered to case study students by language evaluators from 
the Evaluation Project, the COCI ratings were next correlated with the classroom teachers’ 
proficiency ratings of these same students using the Stanford FLOSEM. The top portion of 
Table 7 depicts the correlations between the COCI and FLOSEM ratings for the 1994-95 
school year by language program. All of the correlations, except for the Russian program 
(r=.493), were positive and greater than r=.750 as can be seen in Table 7, and all attained 
statistical significance. Similar correlations computed for the 1995-96 school year resulted in 




[Table 7] Correlation between the COCI and FLOSEM ratings 
for Selected Case Study Students 

(The number of students in the program is recorded in parentheses) 



In the 1994-95 School Year: 



School level 


Language 


Correlation R 


Significance 


Elementary 


Japanese 


NA 


NA 


High School 


together 


0.786 (180) 


p < .0001 




Japanese 


0.756 (84) 


p < .0001 




Chinese 


0.787 (35) 


p < .0001 




Korean 


0.910 (27) 


p < .0001 




Russian 


0.493 (34) 


p < .005 



In the 1995-96 School Year: 



School level 


Language 


Correlation R 


Significance 


Elementary 


Japanese 


0.564 (55) 


p < .0001 


High School 


together 


0.826 (177) 


p < .0001 




Japanese 


0.666 (89) 


p < .0001 




Chinese 


0.709 (31) 


p < .0001 




Korean 


0.934 (30) 


p < .0001 




Russian 


0.748 (27) 


p < .0001 
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similar findings even with students in the Russian programs. These correlations can be 
found in the lower portion of Table 7. 



Summary 

Documentation of language acquisition and growth across instructional time was the 
central concern of this chapter. Using two rating measures — Stanford FLOSEM and Sung 
Language Assessment Questionnaire ~ and the COCI, we found that with few exceptions 
students in all four language programs, at all grade levels and levels of instruction, and on 
each of our language assessment instrument did show improvement within a school year 
(September to May) and across school years. The FLOSEM ratings obtained from teachers 
indicated that teachers saw improvement in their students oral proficiency in the language of 
instruction. Since the teachers knew that their program was being evaluated, it is possible 
that teachers were biased in favor of positively evaluating students’ oral performance on the 
September to May FLOSEM ratings. However, the fact that students’ own self-ratings on 
the FLOSEM were correlated with their teachers lends confidence to the validity of the 
FLOSEM as a measure of oral proficiency and as a technique for assessing growth in the 
language following one and two years of instruction. 

The fact that the students’ self-ratings on the Sung Language Assessment 
Questionnaire also correlated with their FLOSEM provides another layer of evidence that 
both teachers and students recognized the growing linguistic competence shown by students 
learning a LCTL. Finally, the COCI ratings obtained by independent language evaluators 
confirm that case study students across all language programs and levels of instruction do 
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make progress in becoming orally proficient in the language that they are learning in school. 
Again, high and significant correlations between the COCI and teacher ratings on the 
FLOSEM provide concurrent validation of both instruments as easy to use measures to assess 
oral proficiency. 

Finally, as a cautionary note, it is important to mention that there were some 
statistical comparisons which failed to attain significance. This is due to extreme 
heterogeneity of teachers, students, languages and levels of instruction in the data. 
Nonetheless, most students it seems at all grade levels and in most language by level of 
instruction combinations showed progress in learning a LCTL. Also observed in the findings 
was a significant difference of ethnic heritage students on oral proficiency attainment when 
compared to non-ethnic heritage students. This attests to the linguistic advantage that ethnic 
heritage students may have in a language program if they already know the language prior to 
instruction and/or if they are able to obtain assistance in the language from someone other 
than the teacher. We will address issues of ethnic heritage background in the results chapters 



that follow. 
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Chapter 5 

Student and Parent Questionnaire Survey Results 

The student and parent questionnaire was distributed twice — the first time in 1993 
and again in 1995. A total of 1,217 students and 1,577 parents participated in the survey. 

An important aspect of this data collection effort is that we were able to match 555 students 
and parent questionnaires. In the student data, 178 questionnaires were collected from 
elementary school students, 18 from middle school students, and 1,021 from high school 
students. There were 648 male and 560 female students and 9 respondents did not report 
their gender. Student data were collected from 666 students enrolled in Japanese language 
programs, 187 from Chinese (either Cantonese or Mandarin), 195 from Korean, and 169 
from Russian programs. Students’ ethnic heritage language background information was also 
collected. 382 students reported that the language they were studying was their ethnic 
heritage language and 74 students described themselves as mixed-heritage, which meant that 
one of their parents was an ethnic heritage parent. 743 students reported that they were not 
ethnically related to the language they were studying. 18 did not provide ethnic heritage 
information. Table 8 summarizes students’ ethnic background information by language 
program type. The table shows that the highest percentage of ethnic heritage students were 
enrolled in the Chinese (56%) and Korean (93%) programs. 

In the parent questionnaire data, 652 respondents were elementary school parents, 19 
were middle school and 906 were high school parents. There were 546 fathers or male 



[Table 8] 



Number of Students by Their Ethnic Heritage Language Background 
in Each Language Program Type 

(for both elementary and high schools) 



Ethnic Info. 


# & % 


Japanese 


Chinese 


Korean 


Russian 


TOTAL 


Heritage 


number 


94 


105 


181 


2 


382 


percentage 


14.11% 


56.15% 


92.82% 


1.18% 


31.39% 


Mixed 


number 


63 


8 


1 


2 


74 


percentage 


9.46% 


4.28% 


0.51% 


1.18% 


6.08% 


Non-heritage 


number 


496 


71 


13 


163 


743 


percentage 


74.48% 


37.97% 


6.67% 


96.46% 


61.05% 


no response 


number 


13 


3 


0 


2 


18 


percentage 


1.95% 


1.60% 


0% 


1.18% 


1.48% 


TOTAL 




666 


187 


195 


169 


1217 




92 
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guardians and 979 mothers or female guardians. 52 parents did not report their gender. A 
total of 929 were from parents of children who enrolled in Japanese programs, 257 in 
Chinese, 211 in Korean, and 180 in Russian programs. Among the respondents, 1109 
parents answered in the questionnaire in English, 34 in Spanish, 123 in Japanese, 101 in 
Chinese, and 210 in Korean. Parents’ ethnic heritage background information is shown in 
Table 9. Parents of elementary school students between Kindergarten and 3rd grade were 
added to the parent data set since these parents were more likely to be ethnic parents. This 
is why the percentages of ethnic parents for the Japanese and Chinese programs is larger for 
parents than for students. 

Data collected from both students and parents were first analyzed by means of factor 
analysis which is a statistical procedure that permits the user to examine the communality 
between items to determine whether items can be grouped into meaningful clusters or scales. 
Factor analysis allows the user to cluster items based on statistical properties that indicate 
that the items measure a common underlying characteristic such as motivation to learn a new 
language. Once a common cluster of items is identified statistically through factor analysis, 
then the items can be organized into scales for the purpose of contrasting different groups of 
respondents on the cluster in question. The technique allows the user to gain greater 
understanding of diverse items on a questionnaire than can be done if each item is treated 
separately and then comparisons made on each item. The method of principal components 
with varimax rotation with missing data deleted was used in all factor analyses discussed in 
this report. The results of the factor analyses will first be presented, followed by group 
comparisons on the different scales that emerged from the factor analyses. 



[Table 9] 



Number of Parents by Tbeir Ethnic Heritage Language Background 
in Each Language Program Type 

(for both elementary and bigb schools) 



Ethnic Info. 


# & % 


Japanese 


Chinese 


Korean 


Russian 


TOTAL 


Heritage 


number 


298 


174 


194 


4 


670 


percentage 


32.08% 


67.70% 


91.94% 


2.22% 


42 . 48 % 


Mixed 


number 


21 


4 


0 


4 


29 


percentage 


2.26% 


1.56% 


0% 


2.22% 


1 . 84 % 


Non-heritage 


number 


583 


75 


16 


164 


838 


percentage 


62.75% 


29.18% 


7.59% 


91.11% 


53 . 14 % 


no response 


number 


27 


4 


1 


8 


40 


percentage 


2.91% 


1.56% 


0.47% 


4.45% 


2 . 54 % 


TOTAL 




929 


257 


211 


180 


1577 




94 
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Factor Analyses 

Student Questionnaire 

The questionnaire data was factor analyzed using a varimax rotation procedure to 
examine the communality of items. A factor analysis was first computed on Part 1 — the 
"Instrumental/Integrative Motivation" scale for studying the LCTLs. The results of the 
factor analysis revealed that the 8 items loaded onto a single factor (see Table 10). The 
eigen value for this single factor was 2.93 and the computed Cronbach alpha was 0.83, 
indicating that the scale possessed internal consistency with the student population. The 
percent of total variance explained by this single factor was 36.62%. Importantly, the result 
of the factor analysis in this study did not confirm the original Gardner (1985) analysis which 
showed that the 8 items consisted of two separate clusters of 4 items each: one for 
instrumental motivation and the other for integrative motivation. The findings suggest that 
the students in the present study made no distinction between instrumental and integrative 
motivation for learning a LCTL. 

Part 2 of the student questionnaire consisted of 14 items having to do with "Reasons 
for Studying These Specific Languages". The analysis revealed that a three-factor solution 
was best for these items. One item was dropped from the final analysis since it did not load 
on any single factor. Table 11 shows the results of the factor analysis. Five items which 
loaded on Factor 1 constitute a subscale of motivation based on students’ Ethnic Heritage 
(e.g., "It is my heritage language"). One item was not about their own heritage, but their 
friend’s language. Factor 2 consisted of three items which are School-Related items 
pertaining to what students have heard about the language class, teacher, or peers taking the 



[Table 10] 



Factor Analysis of the “Instrumental/Integrated Motivation” 
by Students Enrolled in Fourth through Tvj'elfth Grade 



“Studying Japanese (Chinese, Korean, of Russian) is important because ...” 



ITEM 


Loading 


I will be able to meet and converse with more people. 


.680 


I will be able to participate in cultural activities of 
the language group. 


.664 


It will make me a more knowledgeable person. 


.618 


It will be useful someday to get a job. 


.601 


It will allow me to be more at ease with native speakers 
of the language. 


.595 


I will need the language for my future career. 


.592 


It will enable me to better understand and appreciate art 
and literature of the culture. 


.560 


I will get respect from others if I know a foreign language. 


.515 


Eigen Value 


2.930 


Predicted Variance 


36.622 % 



[Table 11] 



Factor Analysis of the “Reasons for Studying LCTLs” Scale 
by Students Enrolled in Fourth through Twelfth Grade 



“I chose to study Japanese (Chinese, Korean, or Russian) because ...” 

l l JiM Factor Loading 





Factor 1 


Factor2 


Factor3 


It is my heritage language. 


.851 


.000 


-.033 


I want to converse with my relatives. 


.800 


.075 


.093 


I want to converse with rny friends. 


.540 


.182 


.250 


My parents encouraged me to study the language. 


.504 


.100 


.113 


My parents forced me to study the language. 


.427 


.112 


-.089 


I heard that the language classes at my school 
were good. 


.111 


.934 


.156 


I heard that the language teachers at my school 
were good. 


.096 


.779 


.160 


I have friends who decided to study the language. 


.187 


.410 


.106 


I want to study in a foreign country as a 
“study abroad” student. 


.151 


.065 


.649 


I want to travel to countries where the language 
is spoken. 


.306 


.003 


.636 


I thought that studying the language would 
be interesting. 


-.154 


.158 


.601 


I want to use the language for practical situation 
such as ordering at the restaurant. 


.365 


.157 


.425 


I thought it would be more interesting than studying 
French, German, or Spanish. 


-.085 


.202 


.424 


Eigen Value 


3.205 


1.640 


1.088 


Predicted Variance 




45.637 % 
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language class. Factor 3 is composed of five items that describe Personal Interests in 
studying the LCTLs (e.g., "I want to study in a foreign country as a study abroad student"). 
The eigen values for the three factors were respectively 3.205, 1.640, and 1.808. Cronbach 
alphas were also respectable: .78 for Factor 1 ("Ethnic Heritage Factor"); .77 for Factor 2 
("School-Related Factor"); and .69 for Factor 3 ("Personal Interests Factor"). The total 
variance explained by the three factors was 45.64%. 

The results of the factor analysis on the "Language Learning Strategies" scale (Part 3 
of the student questionnaire) showed that again a three-factor solution was best for these 
items (see Table 12). Two items were dropped from the final analysis since they did not 
load on any single factor. The eigen values for the three factors were respectively 3.807, 
1.002, and 0.579. Seven items which loaded on Factor 1 constitute a subscale of Innovative 
Learning Strategies which consisted of creative and active classroom activities ("acting out", 
"making sentences with new words", "creating original dialogues"). Factor 2 consisted of 
four items which are strategies of Outside Classroom Language Use/Practice such as 
"watching movies in the language", and "writing a letter". Factor 3 was composed of two 
items that describe Translation Strategies. The percent of total explained variance 
contributed by the three factors was 41.45%. 

The results of the factor analysis on the "Parental Involvement in Foreign Language 
Study" scale (Part 4 of the student questionnaire) showed that all 9 items loaded onto a single 
factor (see Table 13). The eigen value for the student data was 4.73 and the Cronbach alpha 
was .89. The percent of total variance explained by the single factor for students was 




52.55%. 



[Table 12] 



Factor Analysis of the “Language Learning Strategies” Scale 
by Students Enrolled in Fourth through Twelfth Grade 



fTEM 



I memorize dialogues and say them outloud or 
act them out. 

I create original dialogues in the language. 

I repeat what the teacher says in the language. 

I physically act out new words. 

I use a mental image to learn a new word. 

I create a sentence using new words. 

I use flashcards to learn new vocabulary words 

I read newspapers / books in the language. 

I speak the language outside the classroom 
with my friends. 

I write notes, letters, or compositions in the language. 

I watch TV programs or movies in the language. 

I ask the teacher translate things from the target language 
to English. 

I try to translate things from the language to English. 



Factor Loading 



Factor 1 


Factor2 


Factor3 


.618 


.164 


.164 


.564 


.430 


.009 


.529 


.066 


.337 


.525 


.142 


.058 


.449 


.226 


.132 


.404 


.369 


.113 


.389 


.146 


.144 


.162 


.769 


.162 


.264 


.613 


.063 


.431 


.519 


.050 


.094 


.469 


.133 


.175 


.035 


.898 


.238 


.308 


.460 


3.807 


1.002 


0.579 




41.447 % 





Eigen Value 
Predicted Variance 



[Table 13] 



Factor Analysis of Students’ and Parents’ Responses 
on the “Parental Involvement in Foreign Language Study” Scale 



“My parents (I) ...” 



ITEM 




Loading 




Students 


Parents 


Encourage to practice the language as much 
as possible. 


.845 


.818 


Feel should learn as much of language 
as possible. 


.783 


.758 


Show considerable interest in things related 
to the language class. 


.777 


.765 


Feel should continue studying the language. 


.764 


.751 


Really encourage to study the language. 


.758 


.712 


Think should devote more time to 
language study. 


.716 


.649 


Have stressed importance of the language 
after graduation. 


.706 


.711 


Try to help with language homework. 


.608 


.565 


Urge getting help from teacher if having 
problem with language class. 


.510 


.516 


Eigen Value 


4.730 


4.412 


Predicted Variance 


52.553% 


49.025% 
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Parent Questionnaire 

A factor analysis procedure was also used with the parents’ responses to "Attitudes 
toward Foreign Language Learning" (Part 1 of the parent questionnaire). The analysis 
revealed that all 9 items loaded onto a single factor (see Table 14). The eigen value for this 
single factor was 3.54 and the Cronbach alpha was highly reliable (.86). The percent of total 
variance explained by this scale for attitudes toward foreign language study was 39.33%. 

All 9 items in the second part of the parent questionnaire regarding "Parental 
Involvement" in their child’s foreign language study also loaded onto a single factor (see 
Table 11 again). The eigen value for the parent data was 4.412 and a check for internal 
consistency using the Cronbach alpha (.90) showed that this scale was highly reliable. The 
percent of total variance explained by the single factor for parents was 49.03%. 

Data Analyses for Both Elementary and High Schools 
Student Questionnaire 

Instrumental/Integrative motivation. Using the new scales created from the student 
data to the questionnaire, we then set out to examine whether differences existed on the 
various scales by school level, language program type, and gender. First, we found that the 
instrumental/integrative motivation scale adapted from Gardner was significantly different by 
school level, F (2, 1212) = 10.974,/; < .0001. As can be seen in Figure 5, elementary (M 
= 4.662) and middle school students (M = 5.176) showed significantly higher motivation to 
learn the LCTLs that they were studying than did high school students (M = 4.395). A 
significant difference on motivation also appeared when the four language programs were 



[Table 14] 



Factor Analysis of the Parents’ Responses 
on the “Attitudes Towards Foreign Language Study” 



“It is important to study foreign languages because ...” 

ITEM 



I wish to read newspapers/magazines 
in another language. 

I want to read literature in the original language, 
not translated. 

I wish to learn many foreign languages. 

I enjoy meeting and listening to speakers of 
other languages. 

I wish I could speak another language perfectly. 

Studying a foreign language is enjoyable experience. 

It is important for everyone to learn a foreign language. 

If I stay in another country, I can use the language. 

I want to speak the language, if I visit a foreign country. 



Eigen Value 
Predicted Variance 



Scale 



Loading 

.762 

.725 

.671 

.661 

.599 

.587 

.550 

.545 

.493 

3.540 
39.326 % 



102 



Gardner’s Instrumental/Integrative Motivation 



[Figure 5] 
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compared, F (3, 1211) = 35.590, < .0001. Students in the Russian programs (M = 
3.789) showed significantly lower motivation than students in any of the other language 
programs (see Figure 6). On the other hand, students in the Korean language program 
showed the highest motivation score (M = 4.712) which was significantly higher than 
students in Japanese programs (M = 4.511, p = .037) and also higher (not significantly 
though) than students in Chinese programs (M = 4.525). Female students possessed higher 
motivation (M = 4.501) than their male counterparts (M = 4.398), but the difference was 
not statistically significant. However, there was a significant gender difference found among 
students in the Korean program. Female students in the Korean program (M = 4.786) were 
significantly more motivated than males (M = 4.640), < .0001. 

Heritage-related motivation. There was a significant school level difference on 
heritage-related motivation to study the LCTLs, F (2, 1212) = 89.279, < .0001. Like the 
Gardner’s motivation scale, students from elementary (M = 3.430) and middle school (M = 
4.097) showed significantly higher heritage-related motivation than did high school students 
(M = 1.884), < .0001 (see Figure 7). A significant difference of heritage-related 
motivation was also noticed among the four language programs, F (3, 1211) = 127.900, 

< .0001. Heritage-related motivation was significantly different between any of the two 
language programs (see Figure 8). Students in the Korean program (M = 3.431) showed the 
highest heritage-related motivation, followed by students in Chinese programs (M = 3.010), 
then Japanese program students (M = 1.848). Students in Russian programs scored lowest 
on the heritage-related motivation scale (M = 0.852). As noted in Figure 9, female students 
showed higher heritage-related motivation (M = 2.341) than their male counterpart (M = 



[Figure 6] 
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[Figure 7] 
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[Figure 8] 
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1.974) and the difference was statistically significant, F (1, 1204) = 14.783, p < .0001. 
Regarding the heritage-related motivation scale, analyses were also conducted by students 
ethnic heritage background. Students were divided into three group: Heritage Language 
Background, Mixed Heritage [which meant one parent was of the same heritage as the target 
language], and Non-Heritage related groups. A significant difference was found on the 
heritage-related scale, F (2, 1195) = 671.216,/? < .0001 (see Figure 10). Ethnic Heritage 
(M = 3.717) and Mixed Heritage students (M = 3.546) recorded significantly higher 
motivational scores on this variable than did Non-Heritage students (M = 1.202). 

Significant differences by ethnic heritage were found across all four language programs and 
also when similar analyses were computed for gender. That is, female and male Ethnic 
Heritage (and Mixed Heritage) students scored higher on ethnic heritage motivation than 
female and male students who were not of the target language heritage. 

School-related motivation. There was a significant difference between elementary 
and high schools on school -related motivation, F (2, 1210) = 7.104, p = .001 (see Figure 
11). Elementary students showed a significantly higher school related-motivation (M = 
2.850) than did their high school counterparts (M = 2.335). Middle school students 
recorded a very high mean score (M = 2.896) on school-related motivation, but since the 
variance of the group was very large with a small sample size, significant differences were 
not found when compared to elementary or high school students. There was no language 
program type difference in school-related motivation, but there was a significant gender 
difference, F (1, 1202) = 6.040,/? = .014. Interestingly, male students (M = 2.536) 
showed significantly higher school-related motivation than did female students (M = 2.285). 
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Heritage-Related Motivation Score 



[Figure 10] 
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However, there was a significant interaction effect between students’ gender and language 
program type, F (3, 1196) = 3.015,/? = .029 (see Figure 12). The interaction effect was 
due to male students’ higher school-related motivation scores in all the programs, except for 
the Chinese programs. However, in the Chinese language programs, female students showed 
higher school-related motivation than males. 

Personal interest-related motivation. Like the other motivation scales, personal 
interest-related motivation to study these LCTLs among high school students (M = 3.652) 
was significantly lower than elementary (M = 3.963) or middle school students (M = 

4.050), F (2, 1213) = 5.090, p = .006 (see Figure 13). There was also a significant 
language program type difference, F (3, 1212) = 24.971, p < .0001. As can be seen in 
Figure 14, students in the Japanese programs recorded the highest personal interest-related 
motivation score (M = 3.902) and those in the Russian programs showed the lowest score 
(M = 3.002). There was no gender difference on personal interest-related motivation. 

Language learning strategies. Regarding learning strategies, there was a significant 
difference on the first strategy (Innovative Strategies) by school level, F (2, 1204) = 3.610, 
p = .027 (see Figure 15). Middle school students (M = 4.581) expressed that they used 
significantly more innovative learning strategies such as active and creative classroom 
activities than did students at the elementary (M = 3.769) or high (M = 3.864) schools. 
Analysis of the second strategy (Outside Classroom Language Use) resulted in a significant 
program type difference, F (3, 1208) = 21.755,/? < .0001 (see Figure 16). Students in the 
Korean program (M = 4.555) showed significantly higher scores on the learning strategies 
involving outside classroom language practice than did students of the other three programs. 
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On the other hand, students in Russian programs said that their use of outside classroom 
language practice strategies (M = 3.483) was significantly lower than the other three 
programs. There was also a gender difference in the second learning strategy, F (1, 1201) 

= 8.543,/? = .004 (see Figure 17). Female students (M = 4.143) said that they used 
outside classroom language strategies significantly more than male students (M = 3.916) and 
the female students’ higher use of this learning strategy was found in every school level and 
also in every language program. As for Translation Strategies, there was no school level 
difference nor gender difference. However, there was a significant language program type 
difference, F (3, 1201) = 2.682,/? = .046 (see Figure 18). Tukey HSD multiple 
comparisons revealed that a significant difference was found only between students in the 
Chinese and Russian programs. Translation strategies were reported significantly more often 
by students in the Chinese programs (M = 4.541) than by students learning Russian (M = 
4.199). 

Parental involvement. As for students’ perception of their parents’ involvement in 
their language study a significant difference was found by school level, F (2, 1207) = 
78.035,/? < .0001. Elementary (M = 4.714) and middle (M = 5.314) school students 
reported that their parents were involved in their language study much more than was true 
for high school students (M = 3.303), as can be noted in Figure 19. There was also a 
significant program type difference on students’ perception of parental involvement, F (3, 
1206) = 25.680,/? < .0001 (see Figure 20). Students from Korean (M = 4.042) and 
Chinese (M = 3.893) programs said that their parents were involved in their language study 
significantly more than did students in Japanese (M = 3.498) and Russian (M = 2.717) 
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[Figure 18] 
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programs. The difference between students learning Japanese and Russian was also 
statistically significant, p < .0001. There was a significant gender difference on perception 
of parental involvement, F (1, 1199) = 9.616, p = .002 (see Figure 21). Female students 
(M = 3.692) indicated that their parents were much more involved in their language study 
than did male students (M = 3.406). 

Parent Questionnaire 

Attitudes toward foreign language learning. Parents’ attitudes toward foreign 
language learning was significantly higher among parents of elementary (M = 5.265) and 
middle (M = 5.450) school language programs than parents of high school students (M = 
5.013), F (2, 1563) = 22.942, p < .0001 (see Figure 22). There was also a significant 
difference of parents’ attitudes toward foreign language learning by language program type, 
F (3, 1562) = 6.206, p < .0001. Figure 23 showed that parents of students learning 
Russian (M = 4.906) recorded significantly lower scores on attitudes toward foreign 
language learning than parents of students in the other three programs (mean scores were 
5.131 for Japanese, 5.193 for Chinese, and 5.189 for Korean programs). Gender difference 
was also noted on parents’ attitudes toward foreign language learning, F (1, 1520) = 8.268, 
p = .004 (see Figure 24). Mothers (M = 5.169) showed significantly more positive 
attitudes toward foreign language learning than did fathers (M = 5.052). Detailed analysis 
(two-way ANOVA for language program type and parents’ gender) resulted in a significant 
gender difference only among parents of students learning Japanese. Results also revealed a 
difference in parents’ attitudes depending on their ethnic heritage background, F (2, 1532) = 
9.779, p < .0001. As noted in Figure 25, parents whose ethnic background was related to 
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Students' Perception Score on Parental Involvement 



[Figure 21] 
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Parents' Attitudes toward Foreign Language Learning 



[Figure 22] 
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Parents' Attitudes toward Foreign Language Learning 



[Figure 25] 



Difference in Parents’ Attitudes toward Foreign Language Learning 
by Their Ethnic Heritage Background 
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the language that their child was learning showed significantly higher attitudes toward foreign 
language learning (M = 5.214) than non-ethnic parents (M = 5.047). 

Parents’ involvement in language study. Regarding parents’ reported involvement 
in their child’s language study indicated that there was a significant difference by school 
level, F (2, 1563) = 32.919, < .0001 (see Figure 26). Middle school parents showed the 

highest involvement in their child’s foreign language study (M = 5.638), then elementary 
parents (M = 5.058), followed lastly by high school parents (M = 4.67). Paired 
comparisons of all language level combinations were significant. There was also a 
significant difference of parental involvement by language program type, F (3, 1562) = 
62.162, p < .0001. As can be seen in Figure 27, parents of Russian language programs (M 
= 4.034) showed significantly lower scores on parental involvement than those of the other 
three programs and parents of both Chinese (M = 5.118) and Korean (M = 5.269) 
programs showed significantly higher parental involvement than did parents of students 
learning Japanese (M = 4.835). Regarding parental involvement, mothers (M = 4.881) 
generally reported higher involvement than fathers (M = 4.792), but the difference was not 
statistically significance. Parents’ ethnic heritage was related significantly to the factor of 
parental involvement, F (2, 1531) = 73.194, p < .0001 (see Figure 28). Heritage language 
background parents (M = 5.179) and mixed parents (M = 5.106) were significantly more 
involved in their child’s language study than were non-ethnic parents (M = 4.569). 
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[Figure 26] 
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Elementary School Comparisons 

Student Questionnaire 

Elementary school students who participated in the questionnaire survey were 
available for only two language programs, Japanese and Cantonese Chinese language 
programs. A total of 178 elementary school students responded to the student questionnaire 
survey among whom 1 1 1 students were from two Japanese language programs and 67 
students were from one Cantonese program. There were 74 male and 102 female students 
and two did not report their gender. Among the elementary school respondents, 82 students 
answered that they were ethnically related to the language they were studying, 37 students 
reported as mixed-heritage, 54 were non-heritage related students, and five did not report 
their ethnic heritage information. Elementary school students’ responses were analyzed by 
language program type, students’ gender and ethnic heritage background information. 

Instrumental/Integrative motivation. Language program type difference, F (1, 160) 
= 5.683, =.018, and students’ gender difference, F (1, 160) = 5.259, p = .023, were 
found to be statistically significant for instrumental/integrative motivation to study these 
less-commonly taught languages. Figure 29 showed that Japanese elementary school students 
(M = 4.777) had significantly higher motivation than students in the Cantonese program (M 
= 4.360) and female students in both Japanese and Cantonese programs (M = 4.769) 
consistently reported being more highly motivated than did male students in the same 
programs (M = 4.368). There was no ethnic heritage background difference in students’ 
instrumental/integrative motivation to study these languages. Also no interaction effects 
between language program type, students’ gender and ethnic heritage background were 
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found. 

Heritage-related motivation. Interestingly, the factor of Heritage-related motivation 
to study Japanese or Cantonese among elementary school students did not result in a 
significant difference between students in either of these programs. Also there was no 
difference between males and females on this comparison. However, there was a significant 
difference in Heritage-related motivation between heritage background and non-heritage 
students, F (2, 160) = 18.579, < .0001 (see Figure 30). Ethnic heritage background 
students (M = 3.989) and mixed heritage students (M = 3.635) were significantly more 
motivated by heritage language considerations than were non-heritage students (M = 2.526). 

School-related motivation. When analyses of variance were computed on the 
School-related and Personal Interest-related motivation for elementary school students, no 
significant differences emerged for any of the main effects: language program type, students’ 
gender and ethnic heritage background. Moreover, no significant interaction effects were 
found in these analyses. Apparently, only the dimension of ethnic heritage background was 
meaningful for the elementary level students in reasons for learning the target language. 

Language learning strategies. Among the identified learning strategy clusters that 
students use to learn the target language, only one strategy cluster proved to be significant 
and only when ethnic heritage and non-ethnic heritage students were compared. In this 
analysis, it was found that ethnic heritage students were more likely to use the strategy of 
outside classroom language practice, F (2, 170) = 3.797, p = .024. Figure 31 showed that 
ethnic heritage background students (M = 4.370) reported a significantly higher level of 
outside classroom language use than did non-heritage students (M = 3.664),/? = .018. The 
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outside classroom language use, not surprisingly, was due to the fact that the ethnic heritage 
students had parents and/or relatives who could assist with the language acquisition process. 

Parental involvement. Elementary school students’ perception of their parents’ 
involvement in their language study was analyzed by language program type, students’ 
gender and ethnic heritage background information. There was no significant difference in 
students’ perception on parental involvement between Japanese and Cantonese programs, nor 
between male and female students. Interestingly, there was not a difference between heritage 
background and non-heritage students on their perception on parental involvement. 

However, there was a significant interaction effect by language program type and ethnic 
background information, F (2, 160) = 3.099, p = .048 (see Figure 32). While ethnic 
background students in Japanese programs perceived much higher parental involvement in 
their language study than did non-ethnic students, non-ethnic students in the Cantonese 
program perceived much higher parental involvement than did the ethnic heritage students. 
Parent Questionnaire 

Compared to the small number of elementary school students (N = 178), a large 
number of parents at the elementary schools (N = 652) participated in the questionnaire 
survey. The reason of the difference in number of participants was that only students who 
were in the 4th grade or higher participated in the survey while all parents whose child was 
from Kindergarten to 12th grade participated. Among the total of 652 elementary school 
parents, 469 were from Japanese, 157 from Cantonese and 26 from Russian programs. 

There were 202 fathers and 436 mothers. Fourteen parents did not specify their gender. 

363 parents were ethnic heritage parents, 23 were mixed-heritage and 251 were non-heritage 



[Figure 32 ] 



Interaction between Language Program and Students’ Ethnic Background 
on Elementary School Students’ Perception 
of their Parents’ Involvement in their Language Study 
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parents. 

The information from parents of elementary school children who completed the 
attitudes toward foreign language learning and involvement was analyzed by language 
program type, parents’ gender and ethnic heritage background. There was no significant 
difference for parents’ attitudes toward foreign language learning by language program type, 
but there was a significant language program type difference on parental involvement, F (2, 
646) = 26.124, p < .0001. As can be noted in Figure 33, parents of students in the 
Chinese programs (M = 5.309) showed the highest parental involvement, followed by 
parents of students in the Japanese programs (M = 5.027), while parents of students 
studying Russian (M = 4.085) recorded the lowest parental involvement. Paired 
comparisons between parents’ scores in the various programs were all statistically significant 
dXp < .001. However, there was no significant parent gender difference on either attitudes 
or involvement, indicating that mothers and fathers at the elementary school level had similar 
beliefs about learning a second language and involvement with their child’s learn of a LCTL. 
Parents’ ethnic heritage also influenced the extent of involvement in their children’s language 
study, F (2, 633) = 6.271, p = .002 (see Figure 34). Ethnic heritage parents (M = 5.149) 
showed a significantly higher involvement in their child’s language study than did 
non-heritage parents (M = 4.910). However, there was no difference on attitudes toward 
second language learning by parents’ ethnic heritage, indicating that elementary school 
parents’ attitudes toward foreign language learning was not influenced by their ethnic 
heritage. 




140 



Parental Involvement in the Child's Language Study 



[Figure 33] 



Language Program Type Difference 
on Parental Involvement in Their Child’s Language Study 
at the Elementary Schools 



5.50 n 



5.309 



5.25- 



5.027 



5.00 

4.75 

4.50 
4.25 

4.00 

3.75 

3.50 



liT V V Y V 

Y T I 

wT wT ^7^1^ 
^wTwT^T^TwTw 
uT^T wT^7 

> wT^T^T^T^T^ 
wT wTwT 

* wT wT 

mT wT wT^T wT wT. 

wT^T^T wT wT^T. 

■ wTwT^T^T^T^ 
vTwT^T wT wTw7 

* wTwT^T^T^Tw 

wT^T^T wT wT^Ti 
^wT wT^T^T^Tw 

wT^T wTwT^ 
vTwT wTwTwTi 

* wT wT^T wT wTw 
a^T^T wT wTyTwTi 

* wT wTwT wT 

wT ^TwT wTw7 

* wT wT wT wTwT^ 
mT wT wTyTwTw7 

* wT wT wT^TwTw 
vTwT^T^T wT wT« 

wT wT^lwT^ 
mT^T wT wT wT« 

■ wT wT wT^TwTm 
vT^T^T^T wT wTi 

■ wT wT wT^TuTw 

wT^TwT^TwTi 

* wT ^ 1^7 wT w 7 w 
m7 wT w7y7^7w7< 

* wT ^7 w 7 ^ 7 w 7 w 
y I wT w7y7y7^7 I 

* wT^ 7 ^ 7 ^ 7 w 7 w 
^ I wT % m7^7y7w 7 I 
®wT^7 wT w7w7^ 

V • wT^ 7 y 7 y 7 ^ 7 I 
^wT wT w7y7^7w 
mT w7^7y7y7y 7 I 
^wT wT wT w7^7^ 
vT w7^7y7y7 w 7 « 
^wT wT w7y7y7w 

V • wT^ 7 ^ 7 ^ 7 ^ 7 i 
®wT^7^7 wT w7w 
» I w7 wT w7^7w7 1 

■ wTw 7^7 wT ^ 7 w 
vT wT wT w7^7^7 « 

* wT^ 7^7 wT ^ 7 ^ 
y ^ wT^7^7^7^7< 

■ wT^7^7 wT wT ^ 
yT wT %yT ^y7^7^7 4 

* %yT^7^7 %yT wT w 

yT wT w7y7y7^7. 



^ I I I I I I y 

I I I I I 1 

^ I I I I I I y 

I I I ^^ 7 ^^ I 

^ I I I I I I y 

wTw 7 ^ 7 ^ 7 ^ 7 ^1 
^7 w 7 Y 7 ^ 7 w 7 ^ 7 y 
I I I I I I 
^ I I I I I I y 

I I I I I I 

^ I I I I I I y 

I I I I I I 

^ I I I I I I y 

I I I I I I 

^ I I I I I I y 

I I I I I I 

^ I I I I 1 ^^ I y 

I I I I I I 

^ I I I I I I y 

wT^ 7 w 7 wTw 7 w 1 
f I w 7 w 7 wTw 7 w 7 y 
wTw 7 w 7 w 7 w 7 w 1 
^ 7 w 7 w 7 w 7 w 7 w 7 y 
wT w 7 w 7 w 7 w 7 w I 
^ 7 w 7 w 7 w 7 w 7 w 7 y 
wT w 7 w 7 w 7 w 7 w 1 
p 7 w 7 w 7 w 7 w 7 w 7 y 
wTw 7 w 7 w 7 w 7 wi 
f T w 7 w 7 w 7 w 7 w 7 y 
wTw 7 w 7 w 7 w 7 w < 
^ 7 w 7 w 7 w 7 w 7 w 7 y 
wTw 7 w 7 w 7 w 7 w I 
^ 7 w 7 w 7 w 7 w 7 w 7 y 
wTw 7 w 7 w 7 w 7 wl 
^7 w 7 w 7 w 7 w 7 w 7 y 
wTw 7 w 7 w 7 w 7 w^ 
^7 w 7 w 7 w 7 w 7 wTy 
wTwTw 7 w 7 w 7 w' 

rTwTwTw 7 w 7 w 7 y 

w 7 w 7 w 7 w 7 w 7 w ' 

^ I w I w 7 w 7 w 7 wTy 
w * w * wTwTwTw J 
f • w I wTw 7 ^ 7 ^ 7 y 
wT wT wT wT wT wl 
^TwTwTwTwTwTy 
wl wl wl w 7 ^ 7^1 
^ 7 w 7 w 7 w 7 w 7 wTy 
wTwTwT wTwTw^ 
^ 7 w 7 w 7 w 7 w 7 wTy 
wTwTwTwTwTw^ 
M wTwTwTwTwT^ 

wT™ 7 w 7 w 7 wTw' 
^ 7 w 7 wT wT wTwT y 
wT wTwTwTwTw^ 
MwTwTwTwTwTy 

wl wl wl w 7 ^ 7 w^ 
^TwTwTwTwTwTy 
wTw 7 ™ 7 w 7 ^ 7 wl 
^ 7 wT wTwTwTw Vy 
wT w 7 wTwTwTw^ 
^ 7 wT w 7 wTw 7 wTy 
wT wT w 7 %y 7 ^ 7 w' 
^ 7 w 7 wT wT w 7 w 7 y 
wTw 7 w 7 w 7 wTw^ 
^ 7 w 7 wT wTwTwTy 
wTw 7 wTw 7 wTw^ 
^ 7 w 7 w 7 w 7 wTw’v 
wT wT wTw 7 wTw^ 



4.085 



^Y'Y'T'Y'Y* 

^ * w * wT^ 7 ^ 7 ^T 

■ w I w 7 ^ 7 ^ 7 wT^ 
^ * w * wT^ 7 ^T ^T 

■ w • w7^7y7w7^ 

^ * w * wTwTw 7 

* w • wT^ 7 ^ 7 ^ 7 ^ 
^TwTwTw7yTw7 

^T^ 7 ^ 7 ^ 7 ^ 7^7 

■ wT^ 7 ^ 7 ^ 7 ^ 7 ^ 
y I w7^7^7^7yT 

* wT^ 7^7 wTw 7 m 

y ■ w 7 ^ 7 ^ 7^7 ^T 

■ w * wTwTw 7 ^ 7 ^ 

y I w I wTwTw7^7 

* w * wTwTw7^7^ 
y * w * wTwTw7y^ 

■ w • wTwTw 7 ^ 7 ^ 
y * wT w7yTw7^ ■ 
^ w * wTw7^7^7^ 



Japanese Cantonese Russian 



Language Program Type 



Parental Involvement in the Child’s Language Study 



[Figure 34] 



Difference in Parental Involvement in Their Child’s Language Study 

at the Elementary Schools 
by Parents’ Ethnic Heritage Background 



5.201 



5.20 n 



5.149 



5.15 

5.10 

5.05 

5.00 

4.95 

4.90 

4.85 

4.80 



»T wT w* wT^T u 

w* wT^T^T^T ^1 
pT^T wT^TYlYTy 
w* wT^T^T^T 
p 1 I I I I 

I I I I I ^^1 
p I I I I I I ^ 

wT wT wT^T wT 
■T^T^T^T^T^Ty 
I I f f I ^^1 

p 1 I I ^^1^^ I ly 
w * wT wT V * V* 
pT wT wT wT^T wTy 

pT T 

wT wT^T^T^Twl 
p| wT wT w^v 

wT wT wT^T^T^ I 
►1 wTwT w^v 

w* wT wT^T^Ty I 

►T^T^T^T^T^Ty 
^T wT^T^TyTyI 
pT^T 

w* wT^T^T^T^I 
pf ^T \ 

w * wT^T wTwTwl 
» * wT wT wT wT ^Ty 
w*wT wT^T wTwl 
» I wT wT wT^T wTv 
w*wT^T^T^T^l 
r* w* wT ^T ^T w > y 
w>w>wT^T^T™l 
»• w* w* wT^T^Ty 
wT^T wT wT^T^ I 
►T^T^T wT wT^Ty 
wT wT^TyT^T^i 
pT wT^T^T^T wTy 
wT wT ^T wTwT^ I 
pT w T ^T^T wT wTy 
wTw I ^T wT^Twl 
pT^T^T^T^T wTy 
YT^T^T^T wTwI 
pT^T^T^Twl wTy 
w * wT wT w* wTw I 
.1 ^T^T wT wT wTy 
wT^T wT wT wTwl 
«T w* wT wT wTwTy 
wT^T^IwT wTwl 
pT ^T wT wT wTwT y 

^TwI 

>1 wT wT^T^TyT V 
wT 

wT^T^T^T^T y 
yl 'y* wT^l 

pi ^T^T wT^T^T y 
wTw * wT wT wT Y I 
pi ^T wl w 1 w 1 w* V 
w * W * 'y* wTwTwl 

► * w 1 w 1 w 1 w* wTy 

wl wl V* * w * w * 

wl wl wl w 1 w*y 
w * w * wl w * w* 

* * wl wl wTwTy 
y 1 wl wlwl wl w* 

» * wl wl w * w I w * V 

W*W* W*W* 



^T wT^T^TyT 

l^yl^yl^pl^y l^y 
ul^yl^yl^y l^yl^y 1^ 

^wlwlwl wlwlw 

^l^y l^yl^y l^yl^y 1^ 

i 1 1 1 1 1 

^l^y l^yl^yl^y l^^l^ 

i 1 1 1 1 1 

^l^yl^yl^yl^y l^pl^ 

• wlwlwlwl wlw 

^l^y l^y 

I wlwlwlwl wlw 
vTw®wlwlwlwl> 
l^p 

^l^yl^yl^^l^yl^pl^ 

I 1 1 1 1 1 

I 1 1 1 1 1 

I 1 1 1 1 1 

ul^y l^^l^^l^^l^^l^ 
I 1 1 1 1 1 

rwlwT^lwl wlw 

u 1 1 1 1 1 1 ^ 

I wTwTwlwlwlw 

^l^pl^^l^pl^^l^^l^ 

rwTwTwTwlwl w 

^ 1 ^p 1 ^p 1 ^p 1 1 1 ^ 

rwTwlwTwTwl w 

yl^pl^pl^pl^pl^y 1^ 

IwTwlwTwTwlw 

^T w I wl wl wl w 1 ^ 

IwTwTwIwTwTw 

^l^y l^yl^pl^y l^pl^ 

i I I I I I 

^l^yl^y l^yl^y l^^l^ 

( I I I I I 

ul^ul^y l^yl^yl^f 1^ 

I wlwlwlwlwlw 
uTwIwTwIwTwI^ 

I wlwlwTwTwl w 
ylwTwTwTwTwV 
Iwlwlwlwlwlw 
^1 wl wTwl wl w * s 
r wT w* wT w T w l\y 

|pl w I wl w* wTwT^ 

I w* wl wTw I wTw 

jpT wTwTwT w I wl^ 
Tw* w* wl wl w I w 
|pT wl wTwTwT wT^ 

^ w* w* wl wTwTw 
jpT wl w I wl wl w*^ 
f w* w* w I wTwTw 
^T wl wl w I wl w*^ 
Iw* w* wTwTwTw 
uTwT wl w I wTwl^ 

I wTwT wl w I w* w 
^T wl w* W* W*> 

I wTwT wl w I w I w 
ipT wTwT wl wTw M 
l^w* w I wl w* wTw 
Lpl wT wTwT wTw M 

I wTwl wlwlwlw 
Lpl wl w I wl w* wT^ 

I wTwl wlwlwlw 

^T wl w I wTwT wK 
I W* W* wl W* ^ 
^TwTwTwTwTwT^ 
^wTwIwI wlwlw 
V I wl W* wTwT wl ^ 

I wTwTwT wTwTw 
yl wTwTwTwTwT^ 

I wTwT wl w I wl w 
fc,T^TvyTvyTyyTv>,Tj 



4.910 



^w^w^w^w^w^^ 
^Iw* wlwlwlw® 

■wlwlwlwT^T^ 
^Iw* W* wlwl w* 

Iwl wlwlwlw 
^Iw I wlwlwlw^ 
IwlwTwTwl wl^ 
^Iwlwlwlw^w* 

■ W I W I w I w I w^^ 

^Iwl wlwlwlw* 

• W I W I W * W * W * 
^T w * w * w * w I w * 
TwT^T^T^T^T^ 
mT w * w * w * w I W * 
*w * w * w I w I w * ^ 
kylwlwlwlwlwl 
*w * w I w I w I w * ^ 

1^1 wl wlwlwlw" 

I w I w I w *^wT^T^ 
ly I w I w * ^T^T^T 

TwTwTwTwTwT^ 



Ethnic-Heritage 

Group 



Mixed-Heritage 

Group 



Non-Heritage 

Group 



142 



FLAAP HNAL REPORT 



82 



High School Comparisons 

Student Questionnaire 

High school students who participated in the questionnaire survey were available from 
all four language programs: Japanese, Mandarin Chinese, Korean and Russian. A total of 
1,021 high school students responded to the student questionnaire survey among whom 555 
students were from seven Japanese language programs, 102 students were from two 
Mandarin Chinese programs, 195 students from one Korean program and 169 students from 
four Russian programs. There were 568 male and 446 female students and 7 students did not 
report their gender. It was also found that 382 students reported that their ethnic background 
matched the language they were studying. Another 37 students came from mixed ethnic 
heritage families, with one parent from the language background. Finally, 689 students were 
non-heritage, meaning that their ethnicity did not match the language they were studying. 
Thirteen parents failed to report their ethnic background information. Table 15 summarizes 
the ethnic composition for each language program. High school students’ responses were 
analyzed by language program type, students’ gender and ethnic heritage background. 

Instrumental/Integrative motivation. A significant language program type 
difference was obtained on the Instrumental/Integrative motivation to study the LCTLs, F (3, 
1016) = 33.045, p < .0001. As can be noted in Figure 35, students in the Korean program 
(M = 4.712) had significantly higher Motivation than students in the other three language 
programs. On the other hand, students in Russian programs (M = 3.789) showed the least 
Motivation. Students in Japanese (M = 4.462) and Mandarin programs (M = 4.429) were 
similar in their motivation scores which differed significantly on paired comparison tests 




[Table 15] 
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[Figure 35] 
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from students in the Russian programs, but were significantly lower than students in the 
Korean program. There were no gender nor ethnic heritage background differences in 
students’ Instrumental/Integrative Motivation to study these languages among high school 
students. However, the interaction effect between language program type and ethnic heritage 
background, F (3, 963) = 2.362, p = .07, just missed being statistically significant. (For 
this analysis, the mixed-heritage group was deleted because of the small number of mixed- 
heritage participants in the Mandarin, Korean and Russian programs.) Figure 36 showed that 
the trend towards significance was due to the fact that ethnic heritage background was 
significant for the Mandarin Chinese program, F (1, 100) = 7.613, p = .007. In the 
Mandarin programs, ethnic heritage background students (M = 4.732) was significantly 
higher on Instrumental/Integrative motivation than were the non-ethnic students (M = 

4.208), whereas there was no ethnic heritage background difference in any of the other 
programs. Although Figure 36 shows an apparent large difference between the ethnic 
heritage Russian students and their non-heritage counterparts, no test of significance was 
computed because there were 2 ethnic versus 163 non-heritage. students. 

Heritage-related motivation. Regarding heritage-related motivation to study the 
LCTLs among high school students, there was a significant difference by language program 
type, F (3, 1016) = 139.211,/? < .0001; students’ gender, F (1, 1011) = 8.010,/? = .005; 
and ethnic heritage background information, F (2, 1005) = 565.856, p < .0001. As noted 
in Figure 37, high school students in the Korean program (M = 3.431) showed the highest 
heritage-related motivation, followed by students in Mandarin programs (M = 2.455), and 
then Japanese programs (M = 1.549). Students in Russian programs showed the lowest 
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heritage-related motivation (M = 0.852). Paired comparison tests between any two language 
programs resulted in significance (p < .0001). Female high school students showed 
significantly higher heritage-related motivation (M = 2.045) than their male counterparts (M 
= 1.762) (see Figure 38). On the other hand, ethnic heritage background students (M = 
3.622) and mixed heritage students (M = 3.249) always showed a significantly higher 
motivation than did non-heritage students (M = 1.105). This significant difference by 
students’ ethnic heritage background was obtained in all four language programs and with 
both male and female students (see Figure 39). 

School-related motivation. Turning to School-related motivation, analysis revealed a 
significant gender difference, F (1, 1010) = 10.462, p = .001. As can be seen in Figure 
40, high school male students (M = 2.494) had a significantly higher scores on 
School-related motivation than female students (M = 2.138). However, this significant 
gender difference was mainly due to the Japanese language program since there was a 
significant gender difference in School-related motivation for students in Japanese programs, 

F (1, 550) = 11.541, p = .001, whereas there was no gender difference for the other three 
language programs. There was also a significant interaction effect between gender and 
language program type, F (3, 949) = 2.923, p = .033. As can also be seen in Figure 40, 
female students in the Mandarin program showed higher scores than did the male students. 
This pattern was not observed in the other three language programs. Another significant 
difference on the School-related motivation factor was the relationship to ethnic heritage 
background students, F (2, 1004) = 5.542, p = .004. Ethnic heritage students (M = 2.565) 
showed a significantly higher School-related motivation to study the target language than did 
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[Figure 39] 
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non-heritage (M = 2.278) or mixed heritage (M = 1.676) students. As can be seen in 
Figure 41, this finding was due to the Japanese program (dotted line) because there were no 
significant School-related motivation differences found in the other three language programs. 
(The mixed heritage group was not included in the Figure because there was a very small 
number of students in this group.) 

Personal interest-related motivation. Unlike the heritage-related motivation, 
Personal Interest-related motivation to study the LCTLs gave somewhat different results. 
There was a significant language program type difference, F (3, 1016) = 22.186, p < .0001 
(see Figure 42). Students in the Japanese language programs had the highest scores on 
Personal Interest (M = 3.867) when compared to students in the Korean (M = 3.544) and 
Russian (M = 3.002) programs. This means that students enrolled in Japanese programs 
chose to study Japanese because of personal interest considerations, rather than their heritage 
background. The mean motivation score of students in the Russian program was 
significantly lower ip < .01) than any other of the three language programs. Overall, a 
significant gender difference on the Personal Interest-related motivation, F (1, 1005) = 

4.127, p =.042, was found with female students (M = 3.616) having significantly higher 
scores on Personal Interest than male students (M = 3.422). However, there was also a 
significant interaction effect between language program type and students’ gender, F (3, 

1005) = 3.170, p = .024 (see Figure 43). There was a different pattern of gender 
difference on the Personal Interest-related motivation scale. In the Japanese programs, male 
students had a higher motivation than female students, but in the other language programs, 
female students always had higher motivation scores than male students. No apparent reason 
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is obvious for the gender differences on Personal Interest between the various language 
programs. 

Language learning strategies. In an examination of students’ language learning 
strategies, there was no significant difference found for the first learning strategy (use of 
innovative classroom activities) and third strategy (translation) by language program type, 
students’ gender and ethnic heritage background. However, there was a significant 
difference on the second learning strategy (outside classroom language use) by language 
program type, F (3, 1014) = 22.332,/? < .0001; by students’ gender, F (1, 1003) = 8.668, 
p = .003; and by ethnic heritage background, F (2, 1003) = 31.170,/? < .0001. As noted 
in Figure 44, students in the Korean program (M = 4.555) had the highest level of outside 
classroom language use, which was significantly higher than students in Japanese (M = 
3.936) and Russian (M = 3.483) and higher than Mandarin students (M = 4.184), but this 
difference was not significant. Students in the Russian programs showed the lowest level of 
outside classroom language use and paired comparisons showed that the Russian program 
differed significantly from the other three language programs, at /? < .0001. Female 
students (M = 4.174) recorded a significantly higher level of outside classroom language use 
than did male students (M = 3.881) and this gender difference was found in all four 
language programs (see also Figure 44). The significant ethnic heritage background main 
effect was due to the significant difference between the ethnic heritage group (M = 4.505) 
and non-heritage group (M = 3.796). This difference is due most likely to the fact that 
ethnic heritage students have more opportunities to use the target language outside of the 
classroom than did non-heritage counterparts. 




157 



Outside Classroom Language Use Score 



[Figure 44] 



Language Program Type and Gender Difference 
on Learning Strategy of Outside Classroom Language Use 
for High School Students 
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Parental involvement. High school students’ perception of their parents’ 
involvement in their language study was significantly different by language program type, F 
(3, 1011) = 24.188, p < .0001 (see Figure 45). Students in the Korean program saw their 
parents as more involved (M = 4.042) than did students in the other three programs (M = 
3.266 for Japanese programs, M = 3.046 for Chinese programs, and M = 2.717 for 
Russian programs). Paired comparisons showed that Korean language program students’ 
perception of their parents’ involvement in their language study was significantly higher than 
the other three groups. The difference between Japanese and Russian language students was 
also significant. There was also a significant difference on students’ perception of parental 
involvement by their ethnic heritage background, F (2, 1000) = 43.165,/? < .0001 (see 
Figure 46). Both ethnic heritage background (M = 3.964) and mixed heritage (M = 3.836) 
students reported that their parents were more involved in their language study than 
non-heritage students (M = 2.993) and this difference was statistically significant. Female 
students (M = 3.403) also reported higher parental involvement in their language study than 
did male students (M = 3.222), but the difference was not statistically significant. 

Parent Questionnaire 

Parents whose child was enrolled in one of the FLAAP-funded less-commonly taught 
language programs participated in the questionnaire survey. A total of 906 high school 
parents completed the questionnaire. 460 parents were from Japanese language programs, 81 
from Mandarin, 211 from Korean and 154 from Russian programs. There were 339 fathers 
and 529 mothers, but 38 did not report their gender. Among parent respondents, 288 were 
ethnic heritage parents, 6 were mixed and 586 were non-ethnic parents, but 26 did not 
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Students' Perception Score on Parental Involvement 



[Figure 45] 



Program Type Di^erence on High School Students* Perception 
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Students' Perception Score on Parental Involvement 



[Figure 46] 



Difference in High School Students’ Perception 
of their Parents’ Involvement in their Language Study 
by Ethnic Heritage Background 
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specify their ethnic background. Analyses for parents’ attitudes toward foreign language 
learning and parental involvement were carried out for high school language program type, 
parents’ gender and ethnic heritage background. 

Parents’ attitudes toward foreign language learning. There was a significant 
difference on parents’ attitudes by language program type, F (3, 893) = 5.216, p = .001 
(see Figure 47). Parents of students in the Korean language program (M = 5.189) showed 
the most positive attitudes toward foreign language learning, which was followed by parents 
of students in Mandarin (M = 5.005), Japanese (M = 4.983) and Russian (M = 4.864) 
programs. Paired comparisons indicated that parents of students in the Korean program were 
significantly higher on attitudes than parents in Japanese and Russian programs. Parents’ 
attitudes were also significantly different between male and female parents, F (1, 864) = 
8.917, =.003, with mothers (M = 5.081) showing more positive attitudes than fathers (M 
= 4.912). There was no interaction effect between language program type and parents’ 
gender. Parents’ ethnic heritage background significantly influenced parents’ attitudes toward 
foreign language learning, F (2, 876) = 9.716,/? < .0001 (see Figure 48). Ethnic heritage 
parents (M = 5.187) showed significantly more positive attitudes than non-ethnic parents (M 
= 4.933). 

Parents’ involvement in language study. Parental involvement in their child’s 
language study was significantly different by language program type, F (3, 894) = 43.695,/? 
< .0001, with the highest level of parental involvement for parents of students in the Korean 
program (M = 5.269) and the lowest level of involvement for parents of students in Russian 
programs (M = 4.025) (see Figure 49). Involvement level of parents in the Japanese (M = 



Parents’ Attitudes toward Foreign Language Learning 



[Figure 47] 
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Difference in Parents’ Attitudes toward Foreign Language Learning 
by Their Ethnic Heritage Background 
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4.638) and Mandarin (M = 4.627) was similar, but significantly higher {p < .0001) than 
that of Russian parents and significantly lower than that of Korean parents {p < .0001). 
There was no significant gender difference in parental involvement, but parents’ ethnic 
heritage influenced the level of parental involvement, F (2, 876) = 51.040, p < .0001 (see 
Figure 50). Ethnic heritage-related parents (M = 5.186) reported that they were 
significantly more involved in their child’s language study than did non-heritage parents (M 
= 4.423). Involvement level of mixed-heritage parents (M = 4.741) fell midway between 
the other two parent groups, but there were no statistically significant differences. 

Factors Influencing Students’ Language Proficiency 
Proficiency and Student/Parent Variables 

Students’ assessed proficiency in the target language as determined by teacher ratings 
using the Stanford FLOSEM at the end of the 1995-96 school year was analyzed by means of 
a stepwise multiple regression analysis procedure with the following factors entered as 
independent variables: motivation, language learning strategies, parental attitudes toward 
foreign languages, and parental involvement. The elementary and high school data was 
analyzed separately and the results of the elementary school data analyses will be reported 
first followed by the high school results. A stepwise multiple regression analysis was first 
conducted with only student variables (various kinds of motivation, learning strategies and 
their perception of parental involvement) and then re-computed by including both student and 
parent variables (parents’ attitudes and involvement). The value of a stepwise multiple 
regression analysis is that it permits us to examine the importance of numerous variables all 



[Figure 50] 
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operating simultaneously on the dependent measure which in this case is the FLOSEM score 
achieved by students. An important part of a stepwise regression analysis is the ordering of 
variables that contribute to the dependent measure and the total percent of variance accounted 
for by the constellation of variables shown by the regression analysis to be statistically 
significant. 

Elementary school data. Stepwise regression analysis with only the student data 
entered indicated that 3 variables contributed to the outcome measure. The order of the three 
variables was as follows: Gardner’s instrumental/integrative motivation, innovative language 
learning strategies, and outside classroom language use. The regression findings are reported 
in Table 16. It should be noted that use of innovative language learning strategies was 
negatively related to the FLOSEM ratings, meaning that the greater the reported use of 
innovative strategies the lower the FLOSEM rating of a student. The stepwise regression 
output (see Table 16) also shows that the 3 variables together account for 93% of the total 
variance associated with FLOSEM ratings for all elementary level students. 

When the stepwise regression analysis was recomputed by adding the two parent 
related variables (attitudes toward language study and involvement), the findings were 
different (see Table 16). This analysis revealed that parental involvement was the single 
variable that was statistically significant and which by itself predicted 79% of the total 
variance associated with the teacher rating of oral proficiency on the FLOSEM. When a 
similar stepwise regression was computed for only the elementary level students learning 
Japanese, of the 10 variables entered into the analysis only parental involvement emerged as 
a significant predictor of oral proficiency. In addition to being highly significant, the single 



[Table 16] 



Stepwise Regression Analysis 
of Elementary School Students’ Oral Proficiency 



With Student Variables Only: 



Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Gardner’s Motivation 


3.422 


0.748 


0.858 


4.575 


0.000 


Innovative Lg. Use 


-1.606 


0.719 


-0.313 


-2.235 


0.032 


Outside Class Lg. Use 


1.768 


0.701 


0.406 


2.521 


0.017 


N: 37; MULTIPLE R: 0.962 


SQUARED MULTIPLE R: 0.926 




ADJUSTED SQUARED MULTIPLE R: 0.922 








STANDARD ERROR OF ESTIMATE: 


5.345 










ANALYSIS OF VARIANCE FOR MODEL 




SOURCE SUM-OF-SQUARES 


DF 


MEAN-SQUARE 


F-RATIO 


P 


REGRESSION 


12192.682 


3 


4064.227 


142.264 


0.000 


RESIDUAL 


971.318 


34 


28.568 






With Both Student and Parent Variables: 








Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Parental Involvement 


3.147 


0.135 


0.882 


23.288 


0.000 



N; 156; MULTIPLE R: 0.882 SQUARED MULTIPLE R: 0.778 
ADJUSTED SQUARED MULTIPLE R: 0.778 
STANDARD ERROR OF ESTIMATE: 8.581 



SOURCE 


ANALYSIS OF VARIANCE FOR MODEL 
SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO 


P 


REGRESSION 


39935.949 


1 


39935.949 542.344 


0.000 


RESIDUAL 


11413.551 


155 


73.636 
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variable model for the Japanese elementary program accounts for 78% of the total variance. 
The interpretation that follows from this analysis is that for elementary students parental 
involvement is the most important variable in understanding how well these young learners 
will do in their learning of a LCTL as judged by teachers using the FLOSEM. 

High school data. For the high school students, the stepwise multiple regression 
analysis resulted in a more complicated pattern of findings. Table 17 presents the results 
when only the student variables were entered into the analysis. Of the 8 student variables 
entered, 6 emerged in the stepwise regression. The six variables found to be significant 
were: instrumental/integrative motivation, ethnic heritage motivation, school related 
motivation, innovative language learning strategies, outside classroom language use, and 
translation. These 6 variables constitute a good model since they accounted for 86% of the 
variance surrounding oral proficiency as judged by teachers on the FLOSEM. Two of the 
variables — school related motivation and innovative language learning — entered the model 
negatively. This means that students who enroll in a LCTL because of reasons associated 
with a teacher’s reputation, believing the language program to be good, or having friends in 
the class perform less well on oral proficiency at least as judged by teachers. Similarly, 
students who report use of innovative strategies for learning a language again do less well 
when rated for teachers on oral proficiency. 

When the two parent variables are added to the stepwise regression (see Table 18), 
the results change somewhat. The motivation variable related to instrumental/integrative 
reasons for learning a language does not emerge as a part of the final regression model. 
Further, both of the parent variables — attitudes and involvement - enter significantly into 



[Table 17] 



Stepwise Regression Analysis 
of High School Student’s Oral Proficiency 
with Student Variables Only 



Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Gardner’s Motivation 


1.724 


0.235 


0.568 


7.335 


0.000 


Heritage Motivation 


0.778 


0.156 


0.140 


4.982 


0.000 


School Motivation 


-0.313 


0.140 


-0.066 


-2.232 


0.026 


Innovative Lg. Use 


-0.429 


0.258 


-0.125 


-1.662 


0.097 


Outside Class Lg. Use 


0.894 


0.238 


0.275 


3.760 


0.000 


Translation 


0.455 


0.182 


0.153 


2.497 


0.013 



N; 512; MULTIPLE R; 0.927 SQUARED MULTIPLE R: 0.859 
ADJUSTED SQUARED MULTIPLE R: 0.857 
STANDARD ERROR OF ESTIMATE: 5. 1 70 



ANALYSIS OF VARIANCE FOR MODEL 



SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P 



REGRESSION 82047.704 

RESIDUAL 13522.546 



6 13674.617 511.690 0.000 

506 26.724 
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[Table 18] 



Stepwise Regression Analysis 
of High School Student’s Oral Proficiency 
with Both Student and Parent Variables 



Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Heritage Motivation 


0.567 


0.168 


0.114 


3.370 


0.001 


School Motivation 


-0.454 


0.146 


-0.107 


-3.099 


0.002 


Innovative Lg. Use 


-0.718 


0.262 


-0.230 


-2.740 


0.006 


Outside Class Lg. Use 


0.919 


0.228 


0.311 


4.026 


0.000 


Translation 


0.401 


0.190 


0.149 


2.108 


0.036 


Parent Attitudes 


0.996 


0.240 


0.409 


4.147 


0.000 


Parental Involvement 


0.758 


0.248 


0.298 


3.061 


0.002 



N: 341; MULTIPLE R: 0.939 SQUARED MULTIPLE R; 0.881 
ADJUSTED SQUARED MULTIPLE R: 0.879 
STANDARD ERROR OF ESTIMATE: 4.292 



ANALYSIS OF VARIANCE FOR MODEL 



SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P 



REGRESSION 45673.941 

RESIDUAL 6151.809 



7 6524.849 354.253 0.000 

334 18.419 
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the regression model. The fact that 88% of the variance is accounted for by the 7 variable 
model attests to the adequacy of the regression outcome for high school students. As shown 
in Table 18, school related-motivation and innovative language learning strategies continue to 
be negatively correlated with oral proficiency in the regression model. 

Separate regression analyses were next computed for each high school language 
program separately. These analyses were first done without the two parent variables and 
then again with the parent variables added to the regressions. In the case of all four 
language programs, the analyses which combined student and parent variables resulted in the 
best models. Thus, only the regressions which included both student and parent variables 
will be summarized. 

Table 19 presents the regression output for each of the high school language 
programs. For students studying Japanese, the best regression solution consisted of four 
variables: school related-motivation, personal interest motivation, translation, and parents’ 
attitudes toward language study. Together these four variables account for 91 % of the 
variance surrounding FLOSEM proficiency ratings. Table 19 also shows the results of the 
regression analysis for students studying Mandarin. Here only two variables (Gardner’s 
instrumental/integrative motivation and parental involvement) comprised the regression 
model. Together these two variables predicted 94% of the variance. It is important to note 
that there was no overlap of variables in the Japanese and Mandarin language programs. 

This means that student learning in these two language programs is due to entirely different 
patterns of influence. 

When the Korean language program was next examined, a third model emerged. For 
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[Table 19] 



Stepwise Regression Analysis by Language Program Type: 
High School Student’s Oral Proficiency 
with Both Student and Parent Variables 



Japanese Program: 



Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


School Motivation 


-0.236 


0.138 


-0.064 


-1.715 


0.088 


Personal Interest Motiv. 


0.760 


0.210 


0.271 


3.612 


0.000 


Translation 


0.318 


0.177 


0.132 


1.798 


0.074 


Parent Attitudes 


1.358 


0.173 


0.615 


7.870 


0.000 


N: 204; MULTIPLE R; 0.952 


SQUARED MULTIPLE 


R: 0.907 




ADJUSTED SQUARED MULTIPLE R: 0.906 








STANDARD ERROR OF ESTIMATE: 


3.427 










ANALYSIS OF VARIANCE FOR MODEL 






SOURCE SUM-OF-SQUARES 


DF 


MEAN-SQUARE 


F-RATIO 


P 


REGRESSION 22921.278 


4 


5730.320 


487.796 


0.000 


RESIDUAL 


2349.472 


200 


11.747 






Mandarin Program: 












Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Gardner’s Motivation 


1.158 


0.624 


0.407 


1.854 


0.076 


Parental Involvement 


1.444 


0.560 


0.567 


2.581 


0.016 



N: 27; MULTIPLE R: 0.967 SQUARED MULTIPLE R: 0.935 

ADJUSTED SQUARED MULTIPLE R: 0.933 
STANDARD ERROR OF ESTIMATE: 3.426 



ANALYSIS OF VARIANCE FOR MODEL 
SUM-OF-SQUARES DF MEAN-SQUARE 
4241.036 2 2120.518 

293.464 25 11.739 



SOURCE 

REGRESSION 

RESIDUAL 



F-RATIO 

180.645 



P 

0.000 



Korean Program: 



Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


School Motivation 


-1.156 


0.457 


-0.195 


-2.531 


0.014 


Outside Class Lg. Use 


2.027 


0.610 


0.568 


3.322 


0.001 


Parental Involvement 


1.516 


0.521 


0.521 


2.909 


0.005 


N: 76; MULTIPLE R: 0.927 SQUARED MULTIPLE R: 0.859 

ADJUSTED SQUARED MULTIPLE R: 0.856 
STANDARD ERROR OF ESTIMATE: 6.062 

ANALYSIS OF VARIANCE FOR MODEL 
SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO 


P 


REGRESSION 


16398.890 


3 


5466.297 


148.751 


0.000 


RESIDUAL 


2682.610 


73 


36.748 






Russian Program: 


Variables 


COEF 


ST ERR 


ST COEF 


T 


P(2 TAIL) 


Translation 


0.544 


0.297 


0.260 


1.830 


0.077 


Parent Attitudes 


1.391 


0.282 


0.700 


4.934 


0.000 



N; 34; MULTIPLE R: 0.942 SQUARED MULTIPLE R: 0.888 

ADJUSTED SQUARED MULTIPLE R: 0.884 
STANDARD ERROR OF ESTIMATE: 3.210 

ANALYSIS OF VARIANCE FOR MODEL 

SOURCE SUM-OF-SQUARES DF MEAN-SQUARE F-RATIO P 

REGRESSION 2609.271 2 1304.635 126.614 0.000 

RESIDUAL 329.729 32 10.304 
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Korean language students, three variables (school related-motivation, outside class language 
use, and parental involvement) constituted the best regression model. Again these findings 
are shown in Table 19. These three variables accounted for 86% of the variance. The final 
regression analysis was performed on the Russian program students. Here two variables 
(translation and parent attitudes) fit the model. The total predicted variance associated with 
the two variables for Russian language students was 89%. 

In sum, these stepwise regression analyses provide us with another perspective on the 
factors that contribute to learning of a LCTL. The findings show, for instance, the relative 
importance of parental attitudes and involvement in students’ learning of a language. The 
findings also demonstrate that school related motivation operates in a way that is counter- 
intuitive to what we might expect for a high school language program. That is, a student 
who believes that the teacher and the language program is good, and who has friends in the 
class, does less well as judged by teacher ratings for oral proficiency. Also interesting is the 
fact that reasons for studying a language that are based on instrumental/integrative motivation 
lose their relative importance when the parental variables are added to the stepwise 
regressions. Finally, the fact that so much of the variance associated with the dependent 
variable is accounted for by a small number of student and parent independent variables gives 
us confidence in the data collected as part of this evaluation study. 

Proficiency and Students’ Background Information 

Students’ 1995-96 FLOSEM ratings were also examined by language program type, 
students’ gender and ethnic background. There was a significant difference for high school 
language program type, F (3, 511) = 24.833, p < .0001 (see Figure 51). Students in the 



1995-96 FLOSEM Ratings 



[Figure 51] 



Language Program Type Difference 
in the 1995-96 Teachers’ FLOSEM Ratings 
for High School Students 





Language Program Type 
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Korean program possessed significantly higher oral proficiency (M = 15.725) than did 
students in the other three language programs. On the other hand, students in the Russian 
programs (M = 8.595) showed significantly lower oral proficiency than did students in the 
other three programs. 

There was also a significant gender difference on the FLOSEM scores among high 
school students, F (1, 512) = 7.786, p =.005 (see Figure 52). Female students (M = 
13.321) had significantly higher FLOSEM proficiency ratings than did their male 
counterparts (M = 12.050). Importantly, a significant gender difference was found in every 
language program. 

Not surprisingly, ethnic heritage was also found to be a significant factor for students’ 
oral proficiency for high school students, F (2, 506) = 26.504, p < .0001 (see Figure 53). 
Ethnic heritage students (M = 15.055) attained a significantly higher oral proficiency than 
did non-ethnic students (M = 11.578). This significant ethnic background difference on 
students’ proficiency was also found across all four language programs. 

When FLOSEM ratings were examined for elementary level students, no overall 
gender difference was found. Similarly, when possible gender differences were checked for 
students in the Japanese and Cantonese programs separately none were found. However, 
there was a significant ethnic heritage difference, F (2, 34) = 3.742, p = .034 (see Figure 
54). Like their high school counterparts, FLOSEM proficiency ratings were significantly 
higher for ethnic heritage students (M = 21.417) than they were for non-ethnic students (M 
= 15.583). 
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1995-96 FLOSEM Ratings 



[Figure 52] 



Gender Difference 

in the 1995-96 Teachers’ FLOSEM Ratings 
for High School Students 




Male Female 

Students Students 
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[Figure 53] 



Students’ Ethnic Heritage Background Difference 
in the 1995-96 Teachers’ FLOSEM Ratings 
for High School Students 




Group Group Group 



[Figure 54] 



Students’ Ethnic Heritage Background Difference 
in the 1995-96 Teachers’ FLOSEM Ratings 
for Elementary School Students 
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Summary 

In this chapter, we report on a series of statistical analyses which were carried out to 
explore how ethnic heritage, language program type, grade level of student, and motivational 
and parental factors contribute to learning of a LCTL. The data for the analyses come from 
a questionnaire that was administered to more than 1200 students and 1500 parents. At least 
60 separate findings emerged from our analyses and are discussed in this chapter. Among 
these many findings, we see that elementary students (4th and 5th graders) reported being 
more motivated by instrumental and integrative reasons for learning a LCTL than their older 
high school counterparts. We also observed that students in the Korean language program 
reported the highest level of instrumental/integrative motivation than students in the other 
language programs. 

Ethnic heritage background was also found to be important consideration in LCTL 
teaching and learning. Students who were of the same ethnic background as the language 
studied reported more affinity for the language because of their heritage. This was true for 
all language programs and levels of instruction. Interestingly, this was also true for mixed 
ethnic heritage students (e.g., student studying Japanese with only one parent of Japanese 
heritage). A gender difference was noted in our findings on the importance of ethnic 
heritage-related motivation. Females scored significantly higher on the importance of ethnic 
heritage motivation in learning a LCTL than did males. Ethnic heritage parents also differed 
from non-heritage background parents on attitudes toward language learning and involvement 
in the language study of their children. Regardless of language program type or instructional 
level, ethnic heritage parents were more positive toward second language study and were 
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reported more involvement in their child’s language study than did non-ethnic heritage 
parents. The later finding was also supported indirectly by the fact that "outside classroom 
use of the language" was the strategy which occurred with greatest frequency on the part of 
ethnic background students. 

Collectively the findings reported in this chapter confirm our belief that it is important 
to recognize the importance of motivational and parental background factors in second 
language teaching and learning. The position taken in this evaluation project was to think 
about motivation from several perspectives beginning with Gardner’s (1985) work on 
instrumental and integrative motivation and extending this to include heritage, school, and 
personal-related motives. We believe that by combining our expanded view of motivation 
with heritage background considerations, we have augmented our understanding of how 
students and parents perceive the task of learning a LCTL. 
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Chapter 6 

Teacher and Administrator Perceptions of the Program 

Funding for the 14 projects involved in this evaluation study constituted the first time 
that these projects had received support from outside the local school district for their efforts 
to teach a less-commonly taught language (LCTL). Because of funding received through the 
FLAAP grant, the project coordinators were able to hire teachers and/or teaching aides, 
purchase learning materials such as books, software, and even computers, and take students 
on field trips. However, throughout the period of our interaction with teachers and program 
administrators, many of these individuals indicated that they were very appreciative of the 
FLAAP grant because without it, the school would not have been able to support instruction 
in a LCTL at their school. 

One question that surfaced early on in our work with the program staff at the various 
sites was whether projects would survive after the FLAAP grant terminated. This question 
persisted in one form or another throughout the period of funding because of the tenuous 
nature of the project in many teachers’ and administrators’ view. These individuals 
expressed concern about their lack of support from administrators at the school site or in the 
central administration. 

In order to be better informed about the projects and the challenges experienced by 
teachers and administrators, several efforts were made to obtain additional information from 
the individuals involved at each school site. One such effort involved a 13-item 



FLAAP HNAL REPORT 



98 



questionnaire for teachers and administrators (see Appendix 6) which asked a series of 
questions about their specific language program and the prospect of continuation of the 
program following termination of the grant. The questionnaire was completed by 16 teachers 
and 8 administrators during the final workshop held at Stanford University in Spring 1996. 
Representative of all 14 projects are included in the sample. In addition, discussions were 
also held with some principals and heads of foreign language departments in Spring 1996 to 
assess their impressions of the language program at their respective schools. 

The findings gleaned from the questionnaires and the interviews will be summarized 
next. In order to maintain the confidentiality of our respondents, we will not mention 
specific schools or language programs. However, we will summarize general trends and 
provide some quotes from informants that reflect broadly on most of the programs studied. 

Interest in the Language Program 

On the questionnaire, the respondents indicated that they believed that the program 
was very successful. As support of their belief, 88% of the teachers and administrators felt 
that students were very interested in enrolling in the program and 96% believed that students 
in the program were enthusiastic about learning the language. These same respondents felt 
less support from the parent community, here only 63% maintained that there was good to 
very strong support for the program on the part of parents. 

The major difficulty for most of the teachers and administrators had to do with their 
perception of lack of support for their program. The expression of lack of support was not 
fixed on any certain individual or school administrative unit. Only two respondents believed 
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that they had the support of their local Board of Education and only one person felt that the 
local school site council was supportive of their efforts to introduce a LCTL into the high 
school foreign language curriculum. Somewhat troubling is the fact that only 39% of the 
respondents felt that their principal was highly supportive of their LCTL program. Another 
30% reported an "average" level of support from the principal and 30% indicated low 
support from the principal. Among those teachers whose administrative structure included a 
departmental head, 53% reported that they received a "high" level of support and 33% 
reported "average" support from the chair of their school foreign language department. 

Continuation of the Program After the Grant Ends 

Most of the respondents indicated optimism in one way or another that their program 
would continue after the grant terminated. However, some also expressed concerns that 
without outside funding their program would be adversely impacted. One respondent 
commented: 

"It is extremely hard to predict. I am especially worried about the paraprofessional 
aide’s salary and whether she would be retained by the school. Also, there is even 
less chance for my students to use computers. I am frantically looking for another 
grant to apply for at the moment. " 

Another respondent expressed even more concern about the future of the program: 
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"It will change our school environment completely. Without the grant we will not be 



able to purchase needed materials as the need arises. Also, all the extra-cultural 
events will be terminated. Having our school district support us will be a hard battle. 
The road ahead of us will be a tough one to continue. Our classes have been 
overloaded to accommodate the ‘numbers’. With the grant our classes were kept 
small and open. Without the grant, they are loading our classes to 34 - 36 just so 
they don’t have to hire a new teacher in the program." 

Finally, another respondent indicates how the LCTL will be offered once grant funds 
are no longer available. According to this respondent: 

"Because we have had very small classes, the school will no longer support even one 
full section of the [language]. We will attempt to bus students from several sites to 
one central location with one supervisory teacher and one [language] tutor. " 

In the case of several schools, school administrators had already decided by Spring 
1996 to discontinue the program once funding ended. One respondent described what action 
was being taken by parents to the decision to discontinue the program: 

"The parent group would like to see it continue. They are exploring ways in which to 
make it happen: fund raising projects, volunteers and/or ‘charging’ donations to the 
families to attend the classes." 
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Finally, at one school the decision to terminate the project was followed-up 
immediately by an effort to redirect students to other classes. The respondent explains: 

"The program has been officially ended. All students have been called into the 

counseling office to change their schedules for the 1996-97 year." 

Thus, it is fair to conclude that teachers’ and administrators’ concern about the 
prospects of continuation of their programs were Justified. Most programs have continued in 
some form after termination of the grant, but schools are no longer able to provide all the 
resources to teachers and students that were available while there were funds from the grant. 

Most principals that were interviewed indicated interest in the programs, but also felt 
strapped for resources to provide for the classes. They indicated, for example, that without 
grant support the programs would get what the other foreign language programs get — which 
is "damn little" as one principal summarized the discussion. Several high school 
administrators also worried that the LCTL programs would lose out to the other foreign 
language programs because the Japanese teachers were not as experienced and several were 
not even credentialed to teach in high school. In sum, from the perspective of some 
administrators the LCTL programs were "experimental" and they could avoid being overly 
concerned about language teachers who were not credentialed or who had little teaching 
experience since grant funds were offsetting teachers salaries. For instance, in some 
programs it was necessary to hire native language speakers with little teaching experience to 
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teach the LCTL under the supervision of a master foreign language teacher. However, there 
was some indication that this procedure would not be followed once the grant terminated 
forcing school administrators to hire credentialed teachers or cancel the class. 

Summary 

There was considerable variation in how the FLAAP funded programs were managed 
at each site. However, as discussed in Chapters 3 and 4 students were did acquire the 
LCTLs regardless of school level (elementary vs. high school), limitations of curricular 
materials and related instructional supports (e.g., language laboratory) or whether the 
instructor was credentialed in California to teach. Considering the difficulty of learning a 
LCTL, students did present evidence of learning. It would be a shame if the momentum that 
most of the 14 projects achieved through FLAAP funding be would lost. Our parent 
questionnaire data indicates that parents are generally quite favorable toward the study of 
foreign languages and perceive themselves to be involved in their child’s study of the 
selected language (see Chapter 5). Also the demographics of the State of California indicate 
that ethnic heritage parents and students might be especially motivated to learn a LCTL in a 
public school. Our data regarding ethnic heritage background reinforces the idea that Asian 
students are highly motivated to acquire an ancestral language. In some way these 
considerations must be entered into any equation directed at expanding kindergarten to 
twelfth grade education in the less commonly-taught languages. 
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Portfolios in Foreign Language Classes 



GUIDELINES TO TEACHERS AND STUDENTS 
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The guidelines that follow are based on our examination of 
project portfolios and are intended to enhance the usability of the 
portfolios by students, parents, language teachers, and evaluators. 
In current evaluation practice, portfolios can be used as an 
alternative to more traditional methods of assessing student 
achievement. This practice is called authentic assessment. For 
this form of assessment to be useful, however, it is important that 
student portfolios be prepared with care so that the students' work 
reflects learning in foreign language. 

An integral part of a foreign language portfolio consists of 
audio and video tapes of language samples (e.g., dialogues, skits) 
which can be used to assess proficiency and fluency in the target 
language. Thus, the guidelines included here offer strategies for 
preparing taped language samples. 

We also recognize in offering these guidelines that portfolio 
preparation will vary depending upon the grade level (age) of the 
students. Teachers will be more involved in the actual 
organization and maintenance of portfolios in the lower grade 
levels than in high school where students can be in charge of their 
portfolios. 

The guidelines are offered as recommendations for good 
portfolio development in the foreign language classroom. How these 
guidelines are implemented by the foreign language teacher does 
depend on the grade and/or level of language instruction. 

The guidelines are as follows: 

Planning the Portfolio 

• The purpose of the student language portfolio is to document 
growth in the target language. To do this the portfolio needs 
to contain materials that first establish a baseline of 
student target language ability [at the beginning of the 
school year] . All entries thereafter to the portfolio should 
be useful for showing growth in the target language. 
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• For a portfolio to be useful in assessing foreign language 
development, the collection of materials in a student 
portfolio should begin shortly after the start of classes in 
the Autumn semester and must continue with inclusion of 
materials into the portfolio throughout the school year. 

Organizing the Portfolio 

• A Table of Contents prepared by the teacher or student that 
delineates the contents and the dates of materials entered 
into the portfolio is essential. 

• It is important that all entries in a portfolio be dated and 
numbered chronologically. Both dates and numbers should 
correspond on the table of contents and on the actual item or 
artifact. Each portfolio should contain materials completed 
during the current school year. 

• The teacher or student needs to prepare brief annotations of 
all material placed in a language portfolio. These 
annotations need not be elaborate or lengthy, but they should 
be descriptive of the activity that the student was engaged in 
and which resulted in the production of the artifact. All 
annotations should be placed directly onto the material with 
Post-its. 

• The artifacts that are used to make up the portfolio need to 
be well distributed throughout the academic year. Ideally, 
between one and three items should be added to the portfolio 
at least on a monthly basis. These artifacts should represent 
the full spectrum of classroom learning activities, but 
similar artifacts should appear at the beginning and end of 
the school in order to assess language growth over the nine 
month academic year. 

• A minimum of two artifacts that demonstrate learning of the 
target culture (e.g., report of historical figure or event; 
report of a traditional celebration) should be included in all 
portfolios. Projects can be in English, but students should 
make an effort to do their projects in the target language. 

Student-Prepared Portfolios 

• Whether students can create annotations for their portfolios 
will depend upon the grade level. Generally, students from 
the fourth grade on should be able to add their own 
annotations to their portfolios. Students' annotations should 
include information regarding the reason for including the 
artifact in the portfolio and what they learned from the 
activity. 
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Documenting the Four Langua^^ Skills in Portfolios 

• Portfolios must include materials on students' listening 
comprehension in the target language. These may be audio or 
video tapes, listening comprehension tests, or other artifacts 
that were used in class to obtain check students' ability to 
listen with understanding in the target language. 

• Information on reading comprehension also should be a part of 
students' portfolios. This may include audio or videotapes of 
students narrating a story after it has been read in class. 
In more advanced language classes [or in the upper grades] 
such narration should ideally be longer and, of course, in the 
target language. 

• It is always important to find opportunities to obtain 
spontaneous target language speech samples. We recommend that 
the topic for spontaneous speech be selected before the actual 
taping and that students have the opportunity to think about 
their spontaneous dialogue. However, the speech in this 
spontaneous dialogue segment should be unpracticed and should 
be directed by the teacher. 

• Efforts at using the target language creatively in writing 
stories or narratives are invaluable artifacts for portfolios. 
However, it is important to also focus on the process of 
creative writing and any documentation that includes drafts of 
writing samples are as important as the final written product. 

Planning Audio and Video Tapes 

• The teacher needs to be in charge of all audio and/or 
videotaping, regardless of whether the students write and 
direct the activity. Each tape must have a teacher written 
annotation describing the content and participants (including 
English full name and language name if appropriate) . 

• Prior to the actual taping, the teacher must prepare students 
on the importance of the tape in assessing language learning. 
This means students should speak loudly [but still within a 
normal range] so that they can be clearly heard on the audio 
or videotape by whoever listens or views the tape. 

• Taping should best be done in situations that are as free of 
background noises as possible. Tapes with lots of noise are 
not very useful when trying to judge pronunciation or when 
speakers cannot be heard above the noise. 

• The tape [audio or video] should begin with something 
practical, such as a skit that demonstrates the ability of 
students to participate in an oral presentation. 
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Getting Started in Taping 



• Students need to introduce themselves as clearly as possible 
at the beginning of the taped activity. If there is a 
spokesperson for the group, this person needs to describe the 
activity to be taped and who participated in the writing of 
the activity whether it is a skit, puppet show, or other 
dialogue. 

• In videotaping groups of students, the maximum group size is 
four (4) students. However, the optimal group size for ,a 
videotape is two to three students. In an audio tape, the 
optimal nimber of speakers is two — generally the teacher and 
the student. 

Length of the Audio /Video Tape 

• The actual time that students should spend in the presentation 
of their skit/dialogue/oral presentation depends on the level 
of the students' proficiency (e.g., Japanese I vs. Japanese 
IV) and the number of participants. Generally, students in 
more advanced language classes will present videotaped 
dialogues that are longer and more elaborate than those 
expected of introductory language classes. 

• Each participant in a group videotape should have a minimvun of 
5 or 6 turns [i.e., lines] of dialogue to demonstrate mastery 
of the target language. The skills in particular that are 
important in a videotape are pronunciation, fluency and 
expression in the target language. 

Taping of Classroom Teaching 

• Videotaped material of actual classroom teaching activities is 
also invaluable. This is especially true when students are 
engaged in dialogue types of activities that demonstrate their 
mastery of the target language. Ideally, the teacher will 
begin the taping with a brief introduction in the target 
language of the activity to be taped. When not directly 
involved in the dialogue, students should remain quiet to 
minimize background noise. 

Using a Tripod When Preparing a Video Tape 

• Whenever possible, the video equipment should rest on a tripod 
during the taping sessions. This technique provides a better 
quality videotape. 
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Portfolio: Table of Contents 
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Other refers to project/activity reports, art work, of photos of students’ work. 
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Development and Implementation of Student 
Portfolios in Foreign Language Programs 



Amado M. Padilla 
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Juan C. Aninao 
Dominican College 
Hyekyung Sung 
Stanford University 

ABSTRACT Issues related to portfolio development and use in assessing language learning 
in foreign language education are discussed. Among the issues are “audiences" and "pur- 
poses" of the portfolio. With teachers involved in an evaluation project of less commonly taught 
languages (Japanese, Chinese, Korean, and Russian) representing different educational levels 
(elementary and secondary), we collaborated in the design and implementation of student port- 
folios to examine growth in foreign language proficiency. The contents of portfolios were ana- 
lyzed to determine their usability as a means of assessing foreign language learning. Analysis 
of the portfolios showed that many factors should be considered by teachers in deciding on con- 
tents and objectives of the portfolio. We conclude with a series of recommendations for foreign 
language educators interested in using portfolios to document their students language learn- 
ing progress. 



The wave of educational reforms has brought 
with it an increasing dissatisfaction with tradi- 
tional approaches toward student assessment 
(i.e., standardized testing instruments). Tradi- 
tional assessment has emphasized the mea- 
surement of a given body of defined and 
discrete knowledge as determined by a stu- 
dent’s performance on an objective test (Her- 
man, Aschbacher, and Winters 1992). This 
approach heis often been limited to assess- 
ment of student outcomes at a specific point 
in time and has provided little information 
about teaching and learning processes 
(Thompson 1995). 

Recently, educators have put increasing 
emphasis on a search for alternative forms of 

Amado M. Padilla (Ph.D., Univeisity of New Mexico) is 
Professor of Education at Stanford Univeisity and Chair of 
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Associate with the Foreign Language Assistance Program, 
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measuring the processes inherent in actual 
classroom learning and teaching (Baker 1990; 
Herman, Aschbacher and Winters 1992; and 
Lewis 1992). Alternative methods of assess- 
ment are an integral part of classroom instruc- 
tion and should require students to apply and 
integrate what they know by emphasizing 
complex skills (e.g., ability to analyze, gener- 
alize, and hypothesize) within a relevant, 
meaningful context. These approaches call for 
more student involvement in planning assess- 
ment, interpreting the results of assessment, 
and in self-assessment. One such approach 
that has gained popularity is the use of student 
portfolios. As Calfee and Perfumo (1993) ob- 
serve, portfolios provide “opportunities for a 
revolution in assessment" in a variety of sub- 
ject areas and at different levels of schooling. 

The purpose of this article is to report on 
our effort to develop a system for using stu- 
dent portfolios to document growth over an 
extended period of time in connection with 
students’ learning of a foreign language in a 
formal classroom setting. The students, who 
were at both the elementary and high school 
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levels, were enrolled in courses in less com- 
monly taught foreign languages, namely Chi- 
nese, Japanese, Korean, and Russian in 
California schools. The 14 programs involved 
in this project included nearly 1000 students 
and were part of a larger research project de- 
signed to assess strategies for teaching the less 
commonly taught languages.' 

In order to establish a portfolio assessment 
system that was sensitive to both the learning 
environments of elementary and secondary 
foreign language classrooms and students, a 
literature search on the topic of portfolio as- 
sessment in foreign language education was 
first conducted. Only a few documents that 
were tangentially related to portfolio assess- 
ment in foreign language were identified (e.g.. 
Singer 1993). A more global search of the lit- 
erature revealed that teachers in the subject 
areas of language arts, social studies, math, 
and science (Adam and Hamm 1992; Crowley 
1993; Slater 1994) were much further along in 
developing workable systems of student port- 
folio assessment than in foreign language. 

Although there was scant literature on how 
to develop and implement student portfolios 
as noted by Herman and Winters (1994), 
teachers in all subject areas are mostly in 
agreement that the potential benefits of port- 
folio assessment are numerous. For example, 
Adams and Hamm (1992) state that 

“Portfolios can be used as a tool in the 
classroom to bring students together, to 
discuss ideas, and to provide evidence 
of understanding and the way to apply 
it. Through critical analysis of their 
work — and of their peers — students 
gain insight into other ways of looking at 
a problem.” (103) . 

There are certain important questions that 
come with the use of portfolios, such as: What 
should be placed in the portfolio? How often 
should items be added to the portfolio? Who 
decides what goes into the portfolio? Who 
should be given responsibility for its safekeep- 
ing? What should be done with the portfolio at 
the end of the school year? These are just a 



few of the “nuts and bolts” issues that surface 
when deciding to implement portfolio assess- 
ment in the classroom. 

In the field of foreign language education, 
the advantages of using portfolios are obvi- 
ous: provide students with opportunities to 
display good work, serve as a vehicle for criti- 
cal self-analysis, and demonstrate mastery of a 
foreign language. However, an important dis- 
tinction between a content area such as math 
or science and learning a new language is that 
the learner’s ability to use the language is the 
primary object of study, and students’ current 
use of the target language system and poten- 
tial growth in those abilities over a period of 
time is what is at issue. Thus it becomes cru- 
cial that a student portfolio capture in as 
many ways as possible the learner’s use of the 
target language. Depending on the emphasis 
given to the development of reading skills and 
the ability to compose written products in the 
foreign language, students’ written materials 
will be a part of the student’s portfolio. How- 
ever, in foreign language learning, oral skills 
typically are considered more important than 
reading and writing skills. Since oral language 
use of a target language system in both con- 
trolled and spontaneous situations cannot be 
captured easily through wriUen means, the 
use of audiotapes and videotapes takes on in- 
creased importance. 

Although considerations of what to place in 
the portfolio and how often to do so are im- 
portant, these decisions by necessity have to 
be driven by two fundamental considerations: 
namely, the portfolio’s purposes and audi- 
ences. We will now turn our attention to how 
these primary considerations influence the 
use of student portfolios in the foreign lan- 
guage classroom. 

Purposes of the Portfolio 

There are specific purposes a student port- 
folio can serve in the foreign language class- 
room. The following list of typical purposes by 
no means represents an exhaustive descrip- 
tion of all the possible functions a portfolio 
can fulfill. None of the purposes described 
here is superior to any of the others, and there 
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is no reason why the individual purposes pre- 
sented here cannot be used in combination 
with one another. 

Perhaps a portfolio s greatest potential lies 
in documenting and charting students’ growth 
in proficiency in the four language skills of lis- 
tening, speaking, reading, and writing. If the 
curriculum has been designed in a way that 
allows students to acquire a progressively in- 
creasing base of knowledge and skills, items 
can be placed into the portfolio over time in a 
way that allows anyone looking at the portfo- 
lio’s contents to see increased knowledge and 
sophistication with using vocabulary; to de- 
tect greater accuracy in pronunciation; to 
hear how the learner’s oral production has be- 
come more fluent; and to see growth in using 
the language for written purposes. 

A portfolio can be used to document cer- 
tain kinds of language abilities that standard- 
ized instruments fail to measure. In addition to 
the results of any standardized instruments, * 
such as the ACTFL Oral Proficiency Interview, 
which can help assess the learner’s sponta- 
neous oral production of the tai^et language, 
teachers can place into the students’ portfo- 
lios audiotapes and videotapes that capture 
the learner’s use of the language, which are 
useful in determining the progress or a learn- 
er’s communicative competence. 

A portfolio can also be a place where stu- 
dents may place work that, they believe, 
shows unusual learning and/or ability. During 
the course of a semester or school year, stu- 
dents complete a wide variety of assignments. 

If a student feels particularly proud of a com- 
position she has written in the target language, 
she can place it in her portfolio. If a group of 
students is assigned to write and then perform 
a short skit in the target language and they feel 
a strong sense of ownership of it because they 
created it themselves, it can be videotaped 
and included as part of the students’ portfolios. 

Portfolios can serve as the basis of parent- 
teacher conferences to review student’s 
progress over time throughout the school 
year. As Crowley (1993) points out, “concrete 
excunples can show the...parent the student’s 
performance in more detail than would an ab- 



stract number or letter grade” (544). Parents 
can be more assured of a child’s progress with 
the target language system if they are able to 
view videotapes or listen to audiotapes of 
their child actually using the target language 
over time. Similarly, if parents have the oppor- 
tunity to review compositions the child has 
written in the target language or to compare 
exercise sheets that show how the child has 
steadily come to master more and more of the 
target language’s writing system, they can bet- 
ter understand their child’s progress. 

Finally, the contents of students’ portfolios 
can provide information to teachers that can 
help them make decisions about curriculum. 
If, for example, by reviewing students’ writing 
samples, the teacher finds that many students 
are writing more complex sentences than they 
have been taught, but their attempts have 
grammatical errors, the teacher may decide to 
reprioritize the curriculum plan to cover more 
advanced forms of writing that the students 
appear ready to learn. 

Audiences of the Portfolio 

The term “audience” refers to the person(s) 
the portfolio is intended for. In the same way 
that the main purpose of the student portfolio 
will help determine what into the portfo- 
lio, the teachers’ decision as to the portfolio’s 
primary audience will help guide decisions 
concerning the portfolio’s contents, the fre 
quency of placing items in it, etc. As previ- 
ously mentioned, parents, teachers, students, 
or appropriate administrators Q.e, foreign lan- 
guage department chairpersons, school prin- 
cipals, school curriculum planners) can all 
view a student portfolio though different 
lenses. 

Implicatioiis of ‘"Audience/Purpose’* 
Combinatioiis 

The combination of purpose and audience 
is an essential element in implementing port- 
folio assessment since together they ensure 
that the assessment process is a systematic 
undertaking, not a haphazard process of ran- 
domly “throwing things into a folder or box.” 
When careful consideration is given to the au- 
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dience and purpose of the portfolio, answers 
emerge to such questions as: What should be 
placed in the portfolio? How often should ma- 
terials be placed in the portfolio? Who de- 
cides what goes into the portfolio? Who 
should be given responsibility for its safekeep- 
ing? What should be done with the portfolio at 
the end of the school year? 

As an example, assume that a teacher de- 
cides that the primary purpose of the portfolio 
is to provide a place for the student to keep 
and display work she is particularly proud of, 
and the primary audience for the portfolio is 
the student herself (as opposed to the 
teacher). Given this configuration of purpose 
and audience, teachers open the possibility for 
students to feel a greater investment in the 
learning process. Teachers should consider, 
though, what the contents of the portfolio 
might look like at the end of the period of 
study, in cases where the majority (or all) of 
the control is given to the learner, there is no 
guarantee that at the end of the period of study 
the portfolio’s contents will provide an out- 
sider with evidence of a clear progression of 
skill development and emerging proficiency. 
Consider another example of a teacher who 
decides to make the portfolio parent-focused. 
Here the main purpose could be to place 
items that could be shown as part ot a "Farene 
Teacher Night” conversation about the child’s 
progress. It would be important for a teacher to 
show parents not only evidence of the child’s 
current level of proficiency but also some 
charting of the child’s progress. Thus, one 
might find in this portfolio clusters of related 
items that the teacher has chosen. Groups of 
artifacts would be related to one another be- 
cause they illustrate development, and the 
teacher could thus say to parents, “At the be- 
ginning of the school year^ you can see that 
Elena was able to use the vocabulary to write 
simple sentences, but you can now see that 
she’s using her words to write a story of con- 
nected sentences with some more sophisti- 
cated vocabulary.” 

Finally, suppose that the department chair- 
person of a foreign language program wants 
teachers to focus more of their effort on de- 



veloping students’ communicative compe- 
tence as evidenced in spontaneous oral lan- 
guage situations. Further, the department 
chairperson decides that she wishes to see ev- 
idence of this across individual language pro- 
grams. The audience for the student portfolios 
then becomes the department chairperson, 
and the purpose of the portfolios is to provide 
evidence of a faculty’s implementation of a 
departmental goal. In keeping with this audi- 
ence/purpose combination, teachers could 
decide that portfolios needed to have both 
more and a greater variety of speech samples 
to document growth in communicative com- 
petence. Thus portfolios would probably con- 
tain more video and audiotapes as opposed 
to written documents. 

In sum, the task of deciding upon a combi- 
nation of purpose and audience is an essen- 
tial first step in implementing a system of 
portfolio assessment. Doing this task at the 
onset helps to ensure that the assessment 
process is carried out systematically. In the 
work to be described here, the purpose was to 
establish a portfolio system that could be used 
by foreign language educators at both the ele- 
mentary and secondary levels. We provided 
technical assistance to teachers on strategies 
for developing student portfolios that could 
'uc used to dccumenf a .student’s progress in 
the foreign language. In working with teach- 
ers, our goal was to communicate the impor- 
tance of audience/purpose as they worked 
through the details of establishing portfolios in 
their classrooms. 

Method 

Description of Language Programs 

The programs that we worked with varied 
widely in how foreign language education 
was carried out in the classroom. The pro- 
grams included two elementary total immer- 
sion programs (Chinese and Japanese), an 
elementary Russian culture program, an ele- 
mentary Japanese foreign language program, 
and ten high school foreign language pro- 
grams ranging from introductory to advanced 
classes in Japanese, Chinese, Korean, and 
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Russian. The teachers in the different school 
sites included: teacher aides at the elementary 
level who were proficient in the target lan- 
guage and who did the language instmction; 
credentialed teachers who were native speak- 
ers of the target language, but who were not 
credentialed as foreign language teachers: 
and credentialed teachers who were certified 
to teach a foreign language. 

Procedure Used in Developing Portfolios 
To begin work on portfolios we initiated dis- 
cussions with teachers on the collection of 
student proficiency data. These discussions 
took place early in the school year and in- 
volved the kinds of information that teachers 
could collect to document student proficiency 
in each of the four language skills — listening, 
speaking, reading, and writing. In the case of 
reading and writing, teachers agreed that sam- 
ples of students' writing and tests measuring 
reading comprehension were easily identifi- 
able sources for documenting proficiency in 
these two skills. However, identifying appro- 
priate means for demonstrating listening and 
speaking proficiency was not as easy. We dis- 
cussed with teachers the possibility of using 
audiotapes and videotapes as a means of pro- 
evidence of listening and speaking 
skills.'lssues such as the availability of equip- 
ment and logistics involved in the develop- 
ment of portfolios for large classes soon 
became apparent. As a way of maintaining 
ecological validity, we stressed to teachers that 
it would not be necessary for them to invent 
new types of artifacts (a term commonly used 
for items placed into a student portfolio) for 
purposes of a student’s portfolio. Since teach- 
ers are constantly monitoring student growth 
and progress through both formal and infor- 
mal means, we discussed with teachers the 
kinds of practices they use to assess students’ 
current proficiency levels: the types of materi- 
als and activities used for that purpose, the 
kinds of tests and/or quizzes they utilized, etc. 
Teachers were encouraged to try out different 
alternatives in deciding upon possible portfo- 
lio artifacts. In general, conversations at each 
school proved to be productive, and at all sites 



teachers were excited about the prospect of 
being part of an effort to explore portfolio use 
in foreign langucige education. Teachers were 
asked to submit a written plan explaining the 
kinds of artifacts they intended to use, their ra- 
tionale for choosing that type of artifact, and 
their perceptions of how that artifact could be 
expected to show student ability. 

Six students were chosen from each level in 
each program to serve as case study students 
for this “portfolio experiment." Teachers were 
asked to have these students represent a range 
of abilities and (if possible) to choose an 
equal number of male and female learners. 
The one condition that we insisted on was 
that the case study students not be native 
speakers of the target language. This condition 
was imposed because many ethnic language 
heritage students are frequently enrolled in the 
less commonly taught language programs that 
we worked with, and these students already 
have varying levels of proficiency in the lan- 
guage. However, our focus was on students 
who had no familiarity with the target lan- 
guage prior to enrolling in the class. We re- 
turned to the schools in the middle of the 
school year. These visits were planned to dis- 
cuss two major items. First, we asked teachers 
to share the samples of student work they had 
collected thus Teachers inostly shared 
writing samples, tests, and quizzes. Most of the 
teachers said that they were still in the process 
of collecting samples of student work that 
could document proficiency in listening and 
speaking skills. However, some teachers did 
share samples of audiotapes and videotapes 
of students acting out skits or engaging in 
other activities in the target language. These 
visits revealed that teachers were indeed at- 
tending to the task of collecting student profi- 
ciency data. 

We then discussed with the teachers what 
they had learned about “portfolio purpose and 
audience," while collecting items in student 
portfolios. Teachers were not in complete 
agreement regarding the audience of the port- 
folios. Some felt that the primary audience 
should be the students themselves, while other 
teachers felt that the audience should be 
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teachers. Still other teachers felt that portfolios 
were for parents. However, all teachers agreed 
that the purpose of the portfolio was to docu- 
ment students’ language growth. 

Since the foreign language programs varied 
widely, teachers were allowed to identify a 
purpose and audience for their own foreign 
language program. Teachers were instructed 
that this purpose and audience should guide 
their data collection efforts for the remainder 
of the year. For example, for those teachers 
who felt that the audience for the portfolio 
should be teachers and the purpose to track 
student progress, the process of selecting port- 
folio items would be geared more specifically 
(and thus less randomly) toward including re- 
lated items through which growth could be 
clearly seen. As an example, one teacher of 
Japanese noted her tests on students’ mastery 
of Kanji characters were not necessarily cu- 
mulative. This teacher used two tests, one 
which showed how the student had mastered 
25 characters, the other showing how the stu- 
dent had mastered only the next 25. However, 
these tests could not be used to show growth 
and development because of the disjointed 
nature of the two tests. A similar principle ap- 
plied for choosing items that document oral 
skills. A teacher who included in the portfolio 
a videotape done at the beginning of the year 
that snowed siudems* maslery urceriairi vo- 
cabulary words and language-use functions 
should later include a videotape that shows 
the students’ maintenance of those same 
words and functions and that at the same time 
demonstrates mastery of new words and lin- 
guistic functions. 

It was also believed that an objective mea- 
sure of oral proficiency in the tai^et language 
should be part of the student’s portfolio. Since 
there was no common objective tool for mea- 
suring oral proficiency . in the four less com- 
monly taught languages in our project, we 
developed a matrix that teachers could easily 
use to assess their students’ oral language de- 
velopment. This instrument is called the Stan- 
ford Foreign Language Oral Skills Evaluation 
Matrix (FLOSEM) and was modeled after the 
Student Oral Language Observation Matrix 



(Padilla, Sung, and Aninao 1994). 

Once again, these meetings proved produc- 
tive and stimulating, as reported by some of 
the teachers later in the year. Another impor- 
tant aspect emphasized during the meetings 
was the use of a “table of contents" as part of 
each portfolio. The table of contents explains 
when each artifact was completed, what lan- 
guage skill it documented, and what kind of 
topic and activity it was about. High school 
students were encouraged to complete the 
table of contents for their portfolios while 
teachers were asked to do so for elementary 
students. High school students were also 
asked to provide brief notes about why they 
chose certain artifacts and what they had 
learned from the work. We informed all the 
sites that we expected to receive from each 
program a complete portfolio on each case 
study student (six from each level) at the end 
of the year. Portfolios were received during 
the weeks following the end of the school year. 

Analysis and Results 

After the portfolios were received, native 
speakers of each language examined the con- 
tents of the portfolios. The objective of the 
portfolio review was to determine whether the 
portfolios were useful in documenting lan- 
guage growth over the course of the school 
year. The portfolios cont^pe^ <;hirlent writing 
samples, quizzes, audio and videotapes, and 
special group projects of a written nature 
(e.g., family stories, history of a country, etc.). 
The portfolio examiners in each language 
recorded the contents of each student portfo- 
lio and made comments regarding overall or- 
ganization/structure of the portfolio and each 
artifact’s usefulness in terms of charting 
growth in the target language. In addition, the 
native language reviewers compared the stu- 
dent portfolios with the teacher’s initial port- 
folio plan. 

To coordinate the portfolio review, the ex- 
aminers of each language also frequently met 
as a team throughout the process of studying 
the contents of each portfolio. The purpose of 
these meetings was to compare notes and dis- 
cuss the usefulness of the various artifacts 
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found in the portfolios for assessing growth in 
the four language skills. These discussions 
proved to be useful because they highlighted 
the strengths and weaknesses of the student 
portfolios. For example, a frequently noted 
observation in most of the portfolio evalua- 
tors’ comments was missing dates of when 
various materials (e.g., written samples) were 
completed by the students. Absence of dates 
hindered our analysis of documenting lan- 
guage development across the school year. 

As another example, we found that even 
though all teachers included audio and video- 
tapes in the portfolios, these artifacts varied 
widely in their usefulness for assessing stu- 
dents’ proficiency in the less commonly 
taught language. In some cases, the audio or 
videotapes were of spontaneous speech sam- 
ples that showed the student’s ability to use 
the language communicatively. In other 
cases, the audio or videotapes were dialogues 
that had been memorized by the students and 
from which it was difficult to determine 
whether the students in fact had mastered the 
language beyond the simple rote memoriza- 
tion of the script. In these instances, the audio 
or videotapes were useful in assessing accu- 
racy of pronunciation in the target language, 
but not communicative compeienCc. 

Another problem was that teachers often in- 
cluded only a single audio or videotape for 
the designated student that had generally 
been prepared sometime during the second 
semester. The reason given for this generally 
had to do with student reluctance to be taped 
or the difficulty of locating suitable equipment 
or finding time within the instructional period 
to do the taping. Thus there was no way of 
judging growth in oral competency through 
the school year for some students. In those 
portfolios where there were at least two audio 
or videotapes and where the first had been 
done early in the school year and the second 
late in the year, it was possible to assess the 
student’s growing oral skill in the taiget lan- 
guage. This was especially true in introductory 
foreign language classes (e.g., Japanese Q. 
The portfolios also revealed that teachers at 
the advanced levels of foreign language in- 



struction (e.g., Chinese IV) used a more the- 
matic approach to their instruction. This 
made analyzing the portfolios not only more 
interesting because of the contents but also 
more difficult to determine specific commu- 
nicative growth across the academic year. 

Another concern with respect to the review 
of the audio and video portions of the portfo- 
lios was that oftentimes two or more of the 
case study students collaborated in preparing 
and presenting a skit or puppet show, "rhen 
when the production was taped, it was not 
always possible to identify each speaker or 
what the individual’s contribution was to the 
material. Although these productions were en- 
tertaining and obviously very enjoyable for 
the students, they posed problems from an 
evaluation perspective. "Thus, it was not al- 
ways possible to adequately assess sponta- 
neous communicative competence from 
these group presentations. 

The written samples in the portfolios also 
provided important information regarding the 
growth in the target language; however, many 
teachers provided too few written samples to 
adequately judge writing skill development 
across the academic year. Portfolios that con- 
tained numerous and dated writing samples 
proved '.'ery useful in assessing growth in writ- 
ing ability in the target language. Some teach- 
ers included in the portfolio various drafts of 
students’ writing assignments. These were es- 
pecially valuable since they showed the devel- 
opmental stages of writing in the foreign 
language in response to their teacher’s com- 
ments regarding their attempts to complete the 
written assignment. As part of the process of 
examining the contents of each portfolio, the 
reviewer prepared written commentaries on 
the contents that would be shared later with 
the teacher. These commentaries were first 
presented to the evaluation team during our 
meetings as we discussed the usefulness of the 
contents of all portfolios for documenting lan- 
guage growth in the target languages. 'Rie 
comments were useful in aiding team mem- 
bers to formulate guidelines for foreign lan- 
guage portfolio assessment. In addition, the 
teachers found the commentaries useful in 
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planning for the following year’s student port- 
folios. For example, in the case of one student 
portfolio, the native language reviewer com- 
mented; 

We can see each student’s growth in Ja- 
panese, when we compare audicnt with 
video tapes-2/3. Student’s Japanese has 
developed in terms of syntactic com- 
plexity, functional complexity, utter- 
ance length, number of utterances, etc. 
However, since all of them were memo- 
rized and rehearsed dialogues, they 
don’t necessarily reflect each student’s 
proficiency in Japanese.. .Growth in pro- 
nunciation and intonation is hard to as- 
sess with these limited speech samples. 

Later in the same commentary and refer- 
ring to reading, the evaluator states: 

Translation is used regularly as a means 
for checking comprehension. Although 
sentences for translation became more 
complex, reflecting the target sentence 
structures in each lesson, they didn’t be- 
come longer as the teacher had planned. 

!t might have been irKirfi valuable if the 
student had read a written passage (a 
paragraph, not a sentence) and then had 
this included in the test. 

As for writing, the evaluator says: 

Translation (sentence level) is used to 
demonstrate writing proficiency in 
Japanese. More spontaneous written 
samples would be necessary to chart 
growth in Japanese writing. (Essays are 
used at the higher level in Japanese. Per- 
haps this can start earlier.) 

In sum, these comments illustrate the 
strengths and weaknesses of a particular 
portfolio. The material in the portfolio allows 
the reviewer to determine both the level of 
proficiency attained by the student in the for- 
eign language classroom and the strategies 
used by an exemplary teacher of Japanese to 



assess her students’ acquisition of the lan- 
guage. The commentary also offers feedback 
to the teacher for improving the assessment 
process that will be useful for the multiple 
audiences for which the portfolio is pre- 
pared; that is, the teacher, the student, and 
the student’s parents. 



Conclusions and Reconunendations 

A critical analysis of the contents of portfo- 
lios indicated several important conclusions 
when portfolios were used as part of an as- 
sessment procedure for documenting lan- 
guage development in foreign language 
programs from kindergarten through 12th 
grade. First, it was possible to document 
growth in a foreign language in one academic 
year in most portfolios when appropriate 
items were placed in the portfolio throughout 
the school year. Second, a portfolio was most 
useful as an alternative method of assessment 
when a teacher had a plan that took into ac- 
count purpose and audience. Third, the con- 
tents of a portfolio differed by the grade level 
of the student (e.g., third grade) and by the 
level of difficulty (e.g., third-year high school 
Japanese) of instruction in foreign language. 
Finally, an objective measure Ot oral profi- 
ciency in a foreign language should be part of 
the student portfolio. The Stanford FLOSEM 
was developed and used for this purpose. 

A set of guidelines for implementing portfo- 
lios in foreign language classrooms was com- 
piled and offered to our participating teachers. 
Teachers reported that these guidelines were 
very useful. The guidelines that follow are in- 
tended to enhance the usability of portfolios 
by students, parents, and language teachers. 
How the guidelines offered below are imple- 
mented will depend on the grade and/or level 
of foreign language instruction. The guidelines 
are as follows: 

• The teacher and/or student should pre- 
pare a brief annotation for each item placed 
in a language portfolio. These annotations 
need not be elaborate or lengthy, but they 
should describe the activity that resulted in 
production of the artifact. 
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• It is essential that all entries in a portfolio 
be dated and numbered chronologically. 

• Teachers need to keep in mind that the 
purpose of the student language portfolio is to 
document student growth in the target lan- 
guage. Thus the portfolio needs to contain 
materials that first establish a baseline of stu- 
dents' target language ability at the beginning 
of the school, year. All entries thereafter 
should be directed at showing growth in the 
target language. 

• If possible, teachers must seek out strate- 
gies for including information on students’ lis- 
tening comprehension in the target language. 
This may include audio or videotapes, listen- 
ing comprehension tests, or other artifacts that 
were used in class to obtain information 
about students’ ability to listen with under- 
standing in the target language. 

o Information on reading comprehension 
also should be a part of a student’s portfolio. 
This may include audio or videotapes of stu- 
dents’ retelling a story after it has been read in 
class. In more advanced language classes (or 
in the upper grades) such retelling should ide- 
ally be longer and, of course, in the target lan- 
guage. 

• It is always important to find opportunities 
to obtain spontaneous target language speech 
samples. Again, this may be something that 
could be documented by audio or video- 
tapes. 

• Efforts at using the target language cre- 
atively in writing stories or narratives are in- 
valuable artifacts for portfolios; however, it is 
important to also focus on the process of cre- 
ative writing, and any drafts of writing samples 
are as important as the final written product. 

• The artifacts that are used to make up the 
student’s portfolio need to be well distributed 
throughout the academic year. Ideally, mater- 
ial should be added to the portfolio at least on 
a monthly basis. In this way, documentation 
of language growth across the academic year 
can be collected. 

In addition, we developed another set of 
guidelines especially for using audio and 
videotapes as a means of charting students’ 
oral proficiency development. Guidelines 



were prepared in order to assist teachers to 
maximize the use of audio and videotapes in 
their language classrooms. In using the guide 
lines, it is important to keep in mind the grade 
and/or level of the students because the 
guidelines do not necessarily apply in the 
same way to elementary school language pro- 
grams and to introductory language classes in 
high school. 

• Prior to the actual taping, students should 
be told that it is important that they speak 
loudly, but still within a normal range, so that 
they can be clearly heard on the audio or 
videotape by whoever listens or views the 
tape. 

• Students need to introduce themselves by 
name, and a spokesperson for the group 
being audio or videotaped needs to describe 
the activity to be taped and who participated 
in the writing of the activity, if it is a skit, pup- 
pet show, or other dialogue. 

• In videotaping groups of students, groups 
should not consist of more than six students, 
all of whom have introduced themselves. The 
optimal group size for a videotape is three to 
four students. In an audiotape the optimal 
number of speakers is two — generally the 
teacher and the student or two students. 

• The time that students should spend in the 
presentation of their skit/dialogue/oral pre- 
sentation depends on the level of the students’ 
proficiency (e.g., Japanese I vs. Japanese IV) 
and the number of participants. Generally, stu- 
dents in more advanced language classes 
should present dialogues that are longer and 
more elaborate than those expected of intro- 
ductory language classes. 

• Each participant in a group videotape 
should have a minimum of five or six turns 
(i.e., lines) of dialogue to demonstrate mas- 
tery of the target language. The skills in partic- 
ular that are important in a videotape are 
pronunciation, fluency, and expression in the 
tai^et language. 

• Videotaped material of actual classroom 
settings is also invaluable. This is especially 
true when students are engaged in classroom 
dialogue types of activities that demonstrate 
their mastery of the tai^et language. 
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• Also essential in videotaping is that stu- 
dents have the opportunity to engage in spon- 
taneous target language use. Elxperience has 
shown that many students become fearful 
when being videotaped; thus we recommend 
that teachers foster confidence in students by 
beginning with a skit or dialogue that students 
have prepared. Once this is completed, 
teacher-directed, spontaneous speech can be 
recorded. The topic for spontaneous speech 
may be selected before the actual taping, and 

NOTE 

‘This project was funded by the California De- 
partment of Education to evaluate Model Projects 
in Less Commonly Taught Foreign Languages in 
California Public Schools. We thank Dr. Duarte 
Silva, Executive Director, California Foreign Lan- 
guage Project, Stanford University, for his assis- 
tance. 

REFERENCES 

Adams, D. M., and M. E. Hamm. 1992. “Portfolio As- 
sessment and Social Studies: Collecting, Select- 
ing, and Reflecting on What is Significant." 
Social Education 56:103-105. 

Baker, E. L 1990. What Probably Works in Altema- 
tiue Assessment. Los Angeles: UCLA Graduate 
School of Education; National Center for Re- 
search and Evaluation, Standards, and Student 
Testing. 

Calfee, R. C., and P. Perfumo. 1993. “Student Port- 
folios: Opportunities for a Revolution in Assess- 
ment." Journal of Reading 36:532- 537. 

Crowley, M. L 1993. “Student Mathematics Portfo- 
lio: More Than a Display Case." The Mathematics 
Teacher 86:544-547. 



students may have the opportunity to think 
and practice for spontaneous dialogue 
around the selected topic; however, the 
speech in this spontaneous dialogue segment 
should be unpracticed and should be di- 
rected by the teacher. 

Whenever possible, the video equipment 
should rest on a tripod during the taping ses- 
sions. This technique provides better quality 
videotape. 

Herman, J. L, P. R. Aschbacher, and L. Winters. 
1992. A Practical Guide to Alternative Assess- 
ment. Alexandria, VA: Association for supervi- 
sion and Curriculum Development. 

Herman, J. L, and L Winters. 1994. “Portfolio Re- 
search: A Slim Collection." Educational Leader- 
ship 52,2:48-55. 

Lewis, A. C. 1992. “No Shortcuts for Alternative As- 
sessment." Research and Development Review 
7,4:2-3. 

Padilla, A. M., H. Sung, and J. Aninao. 1994. Stan- 
ford Foreign Language Oral Skills Evaluation 
Matrix (FLOSEM). Unpublished Manuscript, 
Stanford University, School of Education. 

Singer, S., ed. 1993. Portfolio Assessment in Foreign 
Language. Evansville, IN: Evansville-Vander- 
burgh School corporation. 

Slater, T. 1994. “Portfolios in Learning and Assess- 
ment in Physics." The Physics Teacher 32:370- 
373. 

Thompson, L, ed. 1995. K-8 Foreign Language As- 
sessment: A Bibliography. Washington, DC: Cen- 
ter for Applied Linguistics. ERIC Clearinghouse 
on Languages and Linguistics. 



Appendix 3 



The Stanford FLOSEM Instruction 



FLOSEM Instructions 



1 



Stanford Teacher/Student 

Foreign Language Oral Skills Evaluation Matrix (FLOSEM) 

Amado M. Padilla Hyekyung Sung Juan C. Aninao 
Stanford University 

INSTRUCTIONS 
What is the Stanford FLOSEM? 

The Stanford Foreign Language Oral Skills Evaluation Matrix (FLOSEM) is a language 
assessment instrument which offers teachers and possibly students one way of evaluating oral 
proficiency in a foreign language. The Stanford FLOSEM provides a means of arriving at a 
holistic rating of the learner’s overall ability to communicate: that is, to comprehend, to speak, and 
to be understood by others. The assessment process focuses on the five areas of oral proficiency: 
comprehension, fluency, vocabulary, pronunciation, and grammar. 

In its overall design, the Stanford FLOSEM is similar to the "Student Oral Language 
Observation Matrix" (SOLOM), an established assessment tool which many English as a Second 
Language (ESL) practitioners have used to assess and document an ESL student’s oral language 
abilities. The Stanford FLUSEM provides more uciailcu uescriptiGns for each cf the different 
categories in the various levels of oral proficiency. 

How is the Stanford FLOSEM useful? 

The primary purpose of the Stanford FLOSEM is to enable foreign language teachers to assess 
students' oral communicative ability in the target language as well as document the growth of each 
learner’s oral competence in a classroom setting. The Stanford FLOSEM is not an achievement 
test used to measure what a student has mastered within the context of a particular foreign language 
course or program, but rather a more global evaluation of the student's ability to communicate. 
Furthermore, Stanford FLOSEM ratings can provide teachers with information to plan or modify 
their curriculum as well as target specific oral language skills which need improvement. Should 
teachers decide to share the FLOSEM outcomes with their students or have students self-rate 
themselves, students can gain insight into their current strengths and weaknesses in the different 
oral language skills. In addition, FLOSEM ratings can be used to provide parents with information 
on their child's progress in the language. 
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Who should administer the Stanford FLOSEM? 

It is very important that the person administering the Stanford FLOSEM not only be someone 
who is able to observe the learner’s performance across a range of various language learning tasks, 
but also be someone the learner feels comfortable with and who is well-acquainted with the 
student’s capabilities. Since classroom language teachers work with students for several hours 
each week, they are the most informed about students’ communicative abilities. Thus, it is 
recommended that the student’s language teacher administer the Stanford FLOSEM. 

The Stanford FLOSEM may also be used for students in secondary schools to self-rate their 
own oral proficiency. Self-evaluations can often help learners recognize their target language skills 
where more practice or instruction is needed. In addition, the continuum of proficiency levels as 
indicated on the FLOSEM grid can assist students in setting goals to further develop their language 
skills. Some teachers may also want to engage students in a peer-rating activity using the 
FLOSEM. This could entail letting small groups of students rate each others' language ability and 
then sharing these ratings and discussing them. 

How is the Stanford FLOSEM rating scale used? 

Description of the Rating Scale (See attached FLOSEM grid) 

Five categories of language use which the Stanford FLOSEM measures are shown in the first 
column of the matrix: "Comprehension", "Fluency", "Vocabulary", "Pronunciation", and 
"Grammar." For each category, there are six possible levels at which a student can be rated, 

"Level 1” through "Level 6." These six levels represent a continuum of competence, ranging from 
"extremely limited ability" (Level 1) through "native-like ability" (Level 6). A description of the 
general criteria for assessing the student’s ability is provided in each of the matrix cells. Since the 
descriptions in each cell are representative of the universal behavior of language development in 
any foreign language, the rating scale may be used for evaluating the growth of any foreign 
language. 

Instructions 

• Raters must carefully study and understand the content of the descriptions provided in each cell 
of the Stanford FLOSEM. Some key terms used in the descriptions of the FLOSEM are 
defined and provided in the Appendix. It is strongly recommended that the user of the 
FLOSEM study and understand all of the terminology used in the matrix before rating a 
student's oral proficiency. 
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• Raters should observe the learner's performance over a range of language-use tasks and over 
an extended period of at least one month. 

• In determining proficiency levels, raters should compare the student's abilities with those of "a ’ 
native-speaker of the target language who is the same age as the student being rated." An 
obvious advantage that comes with comparing a student's performance to that of "a native 
speaker of the same age" is that it provides a baseline for assessing students in comparable 
language development contexts. 

• Raters should assess learner's oral skills based on the criteria described in each cell and assign 
an appropriate score from " 1 " to "6" on the rating sheet (see attached FLOSEM rating sheet). 

If the rater feels that the learner's proficiency is in-between two levels, a halfway mark may be 
assigned such as "1.5" or "2.5". 

• Scores assigned to each category are then added together to reach an overall oral proficiency 
score for each student. The highest possible score is "30" (native-like in all five categories) 
and the lowest possible score is "5" (very limited proficiency in all categories). 

• It is important to note that a student may vary widely in his/her ability in the different categories 
of the FLOSEM (e.g., comprehension- 2; fluency- 1; vocabulary- 3; etc.). This is illustrated 
below in Table 1. 

• Generally, students' proficiency will increase with the progression of levels in language study 
(see Table 1). However, students enrolled in advanced courses are not necessarily expected to 
receive ratings of "5" or "6" in any or all of the categories. Since foreign language skills take 
time to develop, it is not expected that advanced students will have achieved native-like 
proficiency from studying exclusively in the classroom. 

• The following Table is an example of what the FLOSEM ratings might look like for five 
different students in Japanese 1 through Japanese 5 across the five categories. 



Table 1 



Name 


Level 


Date 


comprehension 


fluency 


vocabulary 


pronunciation 


grammar 


Total 


Susan Doe 


Japanese 1 


6/96 


2 


1 


1.5 


1 


1 


6.5 


Carl Smit 


Japanese 2 


6/96 


2.5 


1 


1.5 


1 


2 


8 


John Lee 


Japanese 3 


6/96 


4 


3 


3.5 


3.5 


3 


17 


Lily Jones 


Japanese 4 


6/96 


4 


3.5 


4 


3.5 


3.5 


18.5 


Rob Roy 


Japanese 5 


6/96 


5 


4 


4.5 


3.5 


5 


24 
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How often should the Stanford FLOSEM be administered? 

The Stanford FLOSEM can be a useful means for assessing a learner’s communicative ability 
on a “pre- and post-test” basis. Thus, it is recommended that, at a minimum, teachers do a 
FLOSEM rating twice in an academic school year: once after a month of initial instruction and 
again at the end of the school year. Arriving at a rating at the midpoint of instruction may also 
prove to be useful. However, since progress with mastering foreign language oral skills takes time 
and practice, attempting to do the Stanford FLOSEM on a high-frequency basis (for example, once 
a week) would be excessive and would not necessarily yield useful information. 



APPENDIX 

Clarification of some terms used in the FLOSEM description 

• Formulaic expressions (See the descriptions in Level 1): There are high-frequency set 

expressions and/or patterns that learners usually encounter first when learning a foreign 
language. These expressions are generally common in most social interactions, such as 
"Hello. My name is .", "How are you today?" and "I'm fine, thank you." 

• Passage (See Levels 2, 3, and 4 of "Comprehension"): Passage refers to any piece of text 
which consists of several lines, but cannot be characterized as a conversation between two or 
more people. In foreign language classes, the use of passages is quite common, whether it be 
for the purpose of having students read them, memorize them, or listen to them for testing of 
comprehension. 

• Grammatical errors (See Levels 2 and 3 of "Grammar"): This category refers to the use of 
language that violates the standardized grammatical rules of the target language. Such errors 
include agreement errors (e.g., *"she like" or *"three book"), mistakes in verb inflections 
(e.g., dropping of required verb endings that signal tense such as *"she walk yesterday" or the 
use of the wrong form of the verb *"she goed") and incorrect word order (e.g., *"three pigs 
little") among others. 

• Word connotations and nuances (See Level 5 of "Vocabulary"): These terms refer to 
socially and culturally-implied word meaning which might not be explicitly defined in 
dictionaries. For example, in English, although "cheap" and "inexpensive" mean the same 
thing, "cheap" carries a connotation that an item, while having a low price, is also inferior in 
quality. 
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• Embedded Structures (See Level 3 of "Comprehension"): This term refers to constructions 
such as (1) relative or subordinate clauses and (2) appositional construction, such as the 
following: 

( 1) The girl that mv brother would like to meet already has a boyfriend. 

(2) My father, a man of great integrity and honesty , is someone I admire a lot. 

As the length of utterances increases through the addition of these types of embedded 
structures, greater demands are placed on the learner's processing skills. Typically, the greater 
the level of skill in the target language, the more capable the learner is of forming and 
comprehending statements and questions with these kinds of embedded structures. The terms 
"complex constructions" and "complex patterns" (see, for example. Levels 4 and 5 of 
"Grammar") refer to sentences which contain these kinds of embedded structures. 

• Abstract topics/ideas and Abstract information (See Level 4 of "Comprehension" and 
Levels 4 and 5 of "Vocabulary"): In the early stages of foreign language learning, students 
generally learn to talk about concrete "here and now" kinds of topics such as what the student 
had for dinner last night. As students move to more advanced levels of study, they acquire the 
skills and vocabulary to be able to express viewpoints on more abstract topics such as what 
he/she thinks about providing food to homeless people. 

• Normal speed (See Levels 4 and 5 of "Comprehension"): This term refers to the rate of 
speech in natural conversations between two or more native speakers. However, when native 
speakers try to communicate with beginning level learners of the target language, they often 
slow down their speech to monitor their interactions and to facilitate the learners’ 
understanding. This adjustment in the rate of speech contrasts with the notion of “normal 
speecd”. 

• Intonation patterns (See Level 4 of "Pronunciation"): The dictionary defines intonation as 
the "pattern or melody of pitch changes in connected speech." As an example, most declarative 
sentences tend to fall in intonation towards the end of the statement, while most questions tend 
to be delivered with rising intonation. Intonation patterns refer to this feature of oral language 
use. 
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Proficiency Scale 



1 



Student Foreign Language Proficiency Questionnaire 



Name: 



Date: 



Language/Level: 



PART 1 Foreign Language Oral Skills Evaluation Matrix (FLOSEM) 



Please read each of the descriptive statements regarding foreign language 
oral proficiency in the following sections. The statements represent a 
wide range of abilities in listening comprehension, fluency, vocabulary, 
pronunciation and grammar. Place a check in the bracket of the level that 
best represents your present ability in each of the sections. If you feel you 
are in between levels, mark the appropriate mid-level slot. 



Listening Comprehension 



Level 1 


[ 


] 


I can understand a limited number of high frequency words and common 
conversational set expressions such as "How are you?" or "My name is-". 


Mid-level 


[ 


] 




Level 2 


[ 


] 


I can understand simple questions and statements in short dialogues or 
passages if it is repeated at slower-than-normal speed. 


Mid-level 


[ 


] 




Level 3 


[ 


] 


I can understand the main point(s) of a short dialogue or passage if spoken 
at slower-than-normal speed. I may need some repetition. 


Mid-level 


[ 


] 




Level 4 


[ 


] 


I can understand most of what is said (all main points and most details) at 
near normal speed. 


Mid-level 


[ 


] 




Level 5 


[ 


] 


I can understand nearly everything at normal speed, although occasional 
repetition may be necessary. 


Mid-level 


[ 


] 




Level 6 


[ 


] 


I can understand everything at normal speed like a native speaker. 
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2 



Fluency 



Level 1 


[ 


] 


I can speak using only short question-answer patterns such as "How are 
you? I am fine, thank you.". 


Mid-level 


[ 


] 




Level 2 


[ 


] 


I can participate in a simple conversation on familiar everyday topics at 
slower-than-normal speed. I must frequently pause during conversation. 


Mid-level 


[ 


] 




Level 3 


[ 


] 


I can express myself using simple language, but make mistakes and pause 
a lot when I try to express complex ideas. 


Mid-level 


[ 


] 




Level 4 


[ 


] 


I can effortlessly express myself at near normal speed. Occasionally, I 
have to slow down when expressing complex ideas and less-common 
expressions. 


Mid-level 


[ 


] 




Level 5 


[ 


] 


I am generally fluent, but occasionally have minor pauses when I search 
for the correct manner of expression. 


Mid-level 


[ 


] 




Level 6 


[ 


] 


I have native-like fluency. 



Vocabulary in Speech 



Level 1 


[ 


] 


I know a limited number of high frequency words and common 
conversational set expressions, (e.g.. How are you? My name is ...) 


Mid-level 


[ 


] 




Level 2 


[ 


] 


I have enough vocabulary to make simple statements and ask questions in 
a simplified conversation. 


Mid-level 


[ 


] 




Level 3 


[ 


] 


I have an adequate working vocabulary. I know some synonyms and can 
express simple ideas in a limited number of different ways. 


Mid-level 


[ 


] 




Level 4 


[ 


] 


I have enough vocabulary to participate in everyday conversation and 
know many alternative ways of expressing simple ideas. 


Mid-level 


[ 


] 




Level 5 


[ 


] 


I have enough vocabulary to participate in more extended discussions on 
various topics. I also know some connotations and nuances of certain 
words and expressions. 


Mid-level 


[ 


] 




Level 6 


[ 


] 


I have an extensive native-like vocabulary. 
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3 



Pronunciation 



Level 1 


[ 


] 


I have difficulty in accurately producing the sounds and sound patterns of 
the language. 


Mid-level 


[ 


] 




Level 2 


[ 


] 


I am beginning to master some sounds and sound patterns, but still have 
difficulty with some of the sounds. 


Mid-level 


[ 


] 




Level 3 


[ 


] 


I can produce most of the sounds and sound patterns, but sometimes need 
to repeat myself to make the utterance more clear. 


Mid-level 


[ 


] 




Level 4 


[ 


] 


My speech is always intelligible, but a definite accent and/or awkward 
intonation patterns are apparent. 


Mid-level 


[ 


] 




Level 5 


[ 


] 


My pronunciation and intonation are near native-like. 


Mid-level 


[ 


] 




Level 6 


[ 


] 


My pronunciation and intonation are exactly like those of a native speaker. 



Grammar in Speech 



Level 1 


[ 


] 


I can only use common conversational set expressions. 


Mid-level 


[ 


] 




Level 2 


[ 


] 


I can produce very basic sentence patterns but with frequent grammatical 
errors. 


Mid-level 


[ 


] 




Level 3 


[ 


] 


I can produce a few complex sentence constructions but with noticeable 
grammatical errors. 


Mid-level 


[ 


] 




Level 4 


[ 


] 


I can speak using a good range of complex patterns and grammatical rules. 
However, occasional errors are still present. 


Mid-level 


[ 


] 




Level 5 


[ 


] 


I have a good command over a large range of complex grammar and errors 
are infrequent. 


Mid-level 


[ 


] 




Level 6 


[ 


] 


I can speak with a native-like command of complex grammatical patterns. 
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BEST COPY AVAILABLE 



Proficiency Scale 



4 



PART II General Foreign Language Proficiency 



Please read the following statements in each of the four areas of listening 
comprehension, reading comprehension, speaking, and writing. Rate 
how well you can perform the various activities in the foreign language 
that you are learning on a scale of 1 (cannot do it at all) to 5 (can do it 
comfortably). Circle the appropriate level. 



SCALE 

1: I cannot do it at all. 

2: I can do it but with great difficulty. 

3: I can do it but with some difficulty. 

4: I can do it fairly well but with occasional difficulty. 

5: I can do it comfortably. 



Listening Comprehension 

1 . I can understand a short message on the 

answering machine. 1 

2. I can watch and understand a t.v. program. 1 

3. I can understand a lecture given by a native 

speaker on a topic that interests me. 1 

4. I can play Bingo. 1 

5. I can understand directions to my friend’s house. 1 

6. I can understand a native speaker describe 

his/her favorite hobby. 1 

7. I can understand a story that the teacher reads 

to us in class. 1 

8. I can understand my teacher's directions in class. 1 

9. I can understand the explanation of the rules 

of a game. 1 

10. I can understand general questions about myself 
and my family. 



2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

3 4 



O 

ERIC 



1 



2 



5 



Proficiency Scale 



Speaking 

1 . I can greet someone. 12 3 4 

2. I can tell someone my summer vacation plans. 12 3 4 

3. I can tell a friend about a t.v. program I recently saw. 12 3 4 

4. I can leave a message on an answering machine. 

(e.g., name, phone number, time, date, reason for calling) 12 3 4 



5. I can converse with a native speaker on any general 

topic using the appropriate language forms. 1 2 3 

6. I can describe my best friend. 1 2 3 

7 . I can introduce myself to other people. 1 2 3 

8. I can explain the rules of my favorite game 

to someone. 1 2 3 



4 

4 

4 

4 



9. I can answer general questions about my family. 12 3 4 

10. I can give someone directions to my house. 12 3 4 



For the next two sections, please rate your ability to read and write in the 
characters of the language you are learning (e.g., Chinese Characters, 
Hangul, Hiragana/Katakana, ). 



Readin£ Comprehension 



1. 


I can read instructions on a test. 


1 


2 


3 


4 


2. 


I can read the names of simple objects. 


1 


2 


3 


4 


3. 


I can read a newspaper. 


1 


2 


3 


4 


4. 


I can read the instructions for a board game. 


1 


2 


3 


4 


5. 


I can read some or all of a popular novel. 


1 


2 


3 


4 



Proficiency Scale 6 



6. 


I can read a letter from a pen-pal. 1 


2 


3 


4 


5 


7. 


I can read magazines with minimal 

use of a dictionary. 1 


2 


3 


4 


5 


8. 


I can read simple sentences in the textbook. 1 


2 


3 


4 


5 


9. 


I can read a short children’s story. 1 


2 


3 


4 


5 


10. 


I can read a note from my teacher. 1 


2 


3 


4 


5 


Writing 

1 . I can list the things in my school bag. 1 


2 


3 


4 


5 


2. 


I can write a review on my favorite movie/book. 1 


2 


3 


4 


5 


3. 


I can write a note to a friend. 1 


2 


3 


4 


5 


4. 


I can write a report on the history of 
a foreign country. 1 


2 


3 


4 


5 


5. 


I can keep a journal. 1 


2 


3 


4 


5 


6. 


I can describe the characteristics of my best friend. 1 


2 


3 


4 


5 


7. 


I can write a letter to a pen-pal. 1 


2 


3 


4 


5 


8. 


I can write about my future plans and 

the reasons for them. 1 


2 


3 


4 


5 


9. 


I can take a simple telephone message. 1 


2 


3 


4 


5 


10. 


I can write an essay expressing my thoughts 
on learning foreign languages in high school. 1 


2 


3 


4 


5 



* If you are studying Mandarin Chinese, how many Chinese characters do you know? Please 

give an approximate number. 

* If you are studying Japanese, how many Kanji characters do you know? Please give an 

approximate number. 
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7 



PART 111 

1. Will you continue to study the foreign language you are presently learning next year 

or in college? Yes No 

2. What was your final grade for this class last semester? (Please circle one) 

A+ A A- B+ B B- C+ C C- D+ D D- F 

3. What grade do you expect to receive for this class this quarter? (Please circle one) 

A+ A A- B+ B B- C+ C C- D+ D D- F 

4. How satisfied are you with the progress you have made in learning this foreign language? 

1 2 3 4 5 



Not Satisfied 



Very Satisfied 



Appendix 5 



Student and Parent Questionnaire 
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PARENT CONSENT FORM FOR STUDENT PARTICIPATION 
Dear Parent, 

Your child is enrolled in a foreign language program which is financially supported by the 
California Department of Education. On behalf of the Department of Education, we would 
like to commend you for your child’s interest in a less-commonly taught language. As 
researchers at Stanford University, we are conducting an evaluation study on programs in 
less-commonly taught languages (Japanese, Chinese, Russian, and Korean) in California 
schools under the request of the California Department of Education. In particular, we are 
interested in the reasons why students decide to enroll in courses in these languages. 

We would appreciate it if you would give permission for your child to participate in our 
study by filling out a questionnaire. 

Also, it is important for you to know that your child’s teacher will not know what your 
child has indicated anywhere on this questionnaire. Your child’s answers to this 
questionnaire have nothing to do with his/her grade. 

Your child’s participation in this information-gathering study is voluntary and will occur 
during class time. Your child has the right to refuse to respond to any item or section. 

Your child’s individual privacy will be maintained in all published and written reports 
resulting from this study. 

If you have any questions about your child’s rights as a study participant, or you are 
dissatisfied at any time with any aspect of this study, you can contact Dr. Hyekyung Sung 
at the “Evaluation Project of Foreign Language Assistance Programs’’ at Stanford 
University, CERAS 203 A, Stanford University, Stanford, CA 94305 (phone: 415-723- 
1827). You may also contact, anonymously if you wish, the Human Subjects Office, 1215 
Welch Road, Modular A, Stanford University, Stanford, CA 94304-5532 (phone: 415- 
723-8666) — you may call collect. 

Thank you for your permission for your child to participate in this study! 



I have read the contents of the above “Consent Form” and understand the purpose of this 
study, and my child’s rights as a study participant. I permit my child to participate in this 
study. 



Printed Name: 



Signature: 



Date: 



Child’s Name: 



BEST COPY AVAILABLE 



STUDENT CONSENT FORM 



Dear Student, 

You are enrolled in a foreign language program which is financially supported by the 
California Department of Education. On behalf of the Department of Education, we would 
like to commend you for your interest in a less-commonly taught language. As researchers 
at Stanford University, we are interested in the reasons why students like you decide to 
enroll in courses in less-commonly taught languages such as Japanese, Chinese, Russian, 
and Korean. 

We would appreciate it if you would take the time to fill out the attached questionnedre. It is 
estimated that filling out this questionnaire will take approximately fifteen to twenty 
minutes. Please think carefully about each item. However, do not spend a lot of time on 
any particular item: give your immediate response. 

This questionnaire is anonymous: that is, we ask that you not sign your 
name anywhere on the questionnaire itself. However, we do ask that you be 
honest in your answers because that is the only way we will be able to get accurate 
information. 

Also, it is important for you to know that your teacher will not know what you have 
indicated anywhere on this questionnaire. Your answers to this questionnaire have nothing 
to do with your grades for your Japanese/Chinese/Russian/Korean class. 

Your participation in this information-gathering study is voluntary. You have the right to 
refuse to respond to any item or section. Your individual privacy will be maintained in all 
published and written reports resulting from this study. 

If you have any questions about your rights as a study participant, or are dissatisfied at any 
time with any aspect of this study, you can contact Dr. Hyekyung Sung at the “Evaluation 
Project of Foreign Language Assistance Programs” at Stanford University, CERAS 203A, 
Stanford University, Stanford, CA 94305 (phone: 415-723-1827). You may also contact, 
anonymously if you wish, the Human Subjects Office, 1215 Welch Road, Modular A, 
Stanford University, Stanford, CA 94304-5532 (phone: 415-723-8666). 

Thank you for your participation in this study! 



I have read the contents of the above “Consent Form” and understand the purpose of this 
study, and my rights as a study participant. I agree to participate in this study. I certify 
that the answers I give will be honest. 



Printed Name: 



Signature: 



Date: 
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STUDENT QUESTIONNAIRE 



DO NOT WRITE YOUR NAME ANYWHERE ON THIS QUESTIONNAIRE! 



GENDER: Male Female AGE: 

What is your e thni c backgroimd? 



Caucasian Native American African-American 

Latino Filipino 

Japanese Russian 

Chinese Other (Please specify: ) 

Korean 

Mixed ancestry (Please specify: ) 



What languages do you speak? 
What is your first language? _ 



What language do you consider to be 
your dominant language? 



If more than one language is spoken in your home, please list them according to how often 
they are used: 

Most-often used language 

2nd most-often used language 

3rd most-often used language 

What level of Russian are you currently enrolled in? 

Russian I Russian II Russian HI Russian IV 



How long have you been smdying Russian? _ years months 

Have you studied any other languages besides Russian ? Yes No. 
If "Yes," please list the languages you have smdied and the 
length of time you smdied them: 

1) years mo nths 

2) years mo nths 



O 



EMC 
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Part 1. Please circle the number that indicates how well you 
agree with each of the statement below. 

6 : strongly agree 

5 : moderately agree 

4 : slightly agree 

3 : neutral (no opinion) 

2: slightly disagree 

1 : moderately disagree 

0: strongly disagree 



EXAMPLE: 

Strongly 
Agree 
6 5 



"Studying Russian is important ..." 

5) because I will need it for 

my future career. 6 5 



Strongly 
Disagree 
4 3 2 1 0 




This student circled the number "2" because he slightly disagreed 
with the idea that he needs to study Russian for his future 
career . 



"Studying Russian is important . . . " 

1) because it will allow me to 

be more at ease with 
Russian-speaking people. 6 

2) because it will allow me to 
meet and converse with more 

and a variety of people. 6 

3) because it will enable me to 
better understand and 
appreciate Russian art 

and literature. 6 

4) because I will be able to 
participate in the activities 

of other cultural groups. 6 

5) because I will need it for 

my future career. 6 

6) because I think it will 

make me a more knowledge- 
able person. 6 
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7) because I think it will 
someday be useful in getting 

a job. 6 5 4 32 

8) because other people will 
respect me more if I have 
knowledge of a foreign 

language . 65432 



Part 2. Listed below are some possible reasons why a student like 
you might choose to study Russian, along with a scale to show how 
strong a reason it might be. Here is an example: 



EXAMPLE: 



A very 
strong 
reason 
in my 
case 

6 5 



4 3 



2 



Not a 
reason 
at all 
in my 
case 
1 0 



"I chose to study Russian because 
my parents encouraged me to . ” 




2 



This student circled the number "4" because in his case, 
somewhat strong reason, but not a very strong reason. 



1 0 
it was a 



Please think about each reason, and then circle the number which 
in>Qic3.t©s how stiron^ s, 3r03,son it w©s in your c 3 .S 0 « 



A very 
strong 
reason 
in my 
case 

6 5 



4 3 2 



Not a 
reason 
at all 
in my 
case 
1 0 



"I chose to study Russian..." 



1) because my parents forced me to. 

2) because my parents encouraged 
me to. 

3) because Russian is my heritage 
language. 



6 5 4 3 2 1 
6 5 4 3 2 1 
6 5 4 3 2 1 



4) because I want to travel to 
countries where Russian is 

spoken. 654321 



0 

0 

0 



0 



"I chose to study Russian. . . " 



A very Not a 

strong reason 

reason at all 

in my in my 

case case 

6 5 4 3 2 1 0 



5) because I can use it for 
practical situations such as 
ordering in a restaurant or 

a market. 6 5 4 3 21 0 

6) because I need to complete a 
foreign language requirement 

to get into college. 6543210 

7) because I thought that studying 
Russian would be an interesting 

thing to do. 6543210 



8) because I thought it would be 
more interesting than studying 
one of the other foreign 
languages such as French, 

German or Spanish. 65432 

9) because I want to spend time 
in a Russian-speaking 
country as an exchange or 
"study abroad" student. 

10) because I have friends who 
decided to take it. 

11) because I heard that the 
Russian classes at my 
school were good. 

12) because I heard that the 
Russian teacher (s) at my 
school is (are) good. 

13) because I have relatives 

who speak Russian, and I 
want to be able to converse 
with them. 6 5 4 3 2 1 0 



6 5 4 3 2 1 0 

6 5 4 3 2 1 0 

6 5 4 3 2 1 0 

6 5 4 3 2 1 0 



14) because I have friends who 
speak Russian, and I want 
to be able to converse 

with them. 6 5 4 3 2 1 0 



Part 3. Listed below are some techniques for studying and 
learning a foreign language. Think about each technique, and then 
circle the number which indicates your opinion on how useful the 
technique for studying and learning is or might be. 



1) Using new Russian words 

in a sentence which I create 
myself . 

2) Using a mental image (a 
"picture in my mind") as 

a way to learn a new word. 

3) Using flashcards to learn 
new vocabulary words . 

4) Physically acting out new 
words . 

5) Trying to find similarities 
between new Russian words 
and the English words they 
correspond to. 

6) Watching Russian TV 
programs and/or movies. 

7) Translating things from 
Russian into English. 

8 ) Having the teacher translate 
things from Russian into 
English. 

9) Having the teacher say 
something in Russian, and 
then the class chorally 
repeats it as a group. 

10) Memorizing dialogues, and 
then saying them out loud 
and/or acting them out. 

11) Creating original (my own) 
dialogues in Russian. 

12) Writing notes, letters or 
compositions in Russian, 



O 

ERIC 



An 

extremely 
useful 
way to 
learn 

6 5 4 



6 5 4 



6 5 4 



6 5 4 



6 5 4 



6 5 4 



6 5 4 



6 5 4 



Not a 
useful 
way to 
learn 
at all 

3 2 10 



3 2 10 



3 2 10 



3 2 10 



3 2 10 



3 2 10 



3 2 10 



2 10 



6 5 4 3 2 1 0 



6 5 4 3 2 1 0 



6 5 4 3 2 1 0 

6 5 4 3 2 1 0 

6 5 4 3 2 1 0 
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13) Reading Russian news- 
papers and/or books. 



4 



3 



2 



14) Speaking Russian outside 
of Russian class with 
other students in my 
Russian class. 

15) Carefully looking over 
the assignments I get back 
from my Russian teacher, 
and then correcting my 
mistakes . 



An 

extremely 
useful 
way to 
learn 

6 5 



Not a 
useful 
way to 
learn 
at all 
1 0 



6 5 4 3 2 1 



0 



6 5 4 3 2 1 



0 



6 5 4 3 



2 10 



questions has to do with the role your 
Studying Russian. Please circle the 
number which tells how true the statement is in your case NOTE- 

about an answer, leave it blank- -do not 

CIRCLE ANY NUMBER. uu jnui 



Absolutely 
true 
in my 
case 

6 5 

1) My parents really encourage me 

to study Russicin. 6 5 

2) My parents try to help me with 

my Russian homework. 6 5 

3) My parents feel that I should 

continue studying Russian. 6 5 

4) My parents think I should devote 

more time to my Russian studies. 6 5 



5) My parents show considerable 
interest in anything to do with 
my Russian class. 

6) My parents encourage me to 
practice my Russian as much as 
possible. 



Not true 
at all 
in my 
case 

4 3 2 1 0 



4 3 2 1 0 



4 3 2 1 0 



4 3 2 1 0 



4 3 2 1 0 



4 3 2 1 0 



4 3 2 1 0 
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Absolutely 
true 
in my 
case 

6 5 4 3 2 


Not true 
at all 
in my 
case 
1 0 


7) My parents have stressed the 
importance Russian will have 

for me when I finish school. 65432 


1 0 


8) My parents feel that I should 
learn as much Russian as 

possible. 6 54 3 2 


1 0 


9) My parents urge me to get 
help from my teacher if I am 
having problems with my 

Russian classes. 65432 


1 0 
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PARENT CONSENT FORM 



Dear Parent, 

Your child is enrolled in a foreign language program which is financially supported by the 
California Department of Education. On behalf of the Department of Education, we would 
like to commend you for your child’s interest in a less-commonly taught language. As 
researchers at Stanford University, we are conducting an evaluation study on programs in 
less-commonly taught languages (Japanese, Chinese, Russian, and Korean) in California 
schools under the request of the California Department of Education. One area we are 
interested in is parents’ attitudes towards foreign language study, and what role they might 
play in supporting their child’s study of a foreign language. 

We would appreciate it if you would take the time to fill out the attached questionnaire. It is 
estimated that filling out this questionnaire will take approximately ten minutes. 

This questionnaire is anonymous: that is, we ask that you not sign your 
name anywhere on the questionnaire itself. This signed consent form will 
be detached from the questionnaire after collection. However, we do ask that 
you answers accurately reflect your feelings about the questions. 

It is also important for you to know that your answers on this questionnaire will not in any 
way affect your child’s grades for his or her foreign language class. 

Your participation in this information-gathering study is voluntary. You have the right to 
refuse to respond to any item or section. Your individual privacy will be maintained in all 
published and written reports resulting from this study. 

If you have any questions about your rights as a study participant, or are dissatisfied at any 
time with any aspect of this study, you can contact Dr. Hyekyung Sung at the “Evaluation 
Project of Foreign Language Assistance Programs’’ at Stanford University, CERAS 203 A, 
Stanford University, Stanford, CA 94305 (phone: 415-723-1827). You may also contact, 
anonymously if you wish, the Human Subjects Office, 1215 Welch Road, Modular A, 
Stanford University, Stanford, CA 94304-5532 (phone: 415-723-8666) — you may call 
collect. 

Thank you for your participation in this study! 



I have read the contents of the above “Consent Form’’ and understand the purpose of this 
study, and my rights as a study participant. I agree to participate in this study. I certify 
that the answers I give will be honest. 



Printed Name: 



Signature: 



Date: 



Child’s Name: 




BEST 
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PARENT QUESTIONNAIRE 



DO NOT WRITE YOUR NAME ANYWHERE ON THIS QUESTIONNAIRE! 



Part 1. Please circle the number corresponding to how well vou 
agree with each of the statements below: 

6 : strongly agree 

5: moderately agree 

4:- slightly agree 
3 : neutral (no opinion) 

2: slightly disagree 

1 : moderately disagree 

0 : strongly disagree 



EXAMPLE: Studying a foreign 
language is a good 
thing to do. 



6 



5 




2 



1 



0 



This parent only slightly agrees with the idea that studying a 
f Isnguage is a good thing to dO/ so she circled 4 . 



1) If I were visiting a foreign 

country, I would like to be 
able to speak the language of 
the people . g 

2) It is important for everyone 

to learn foreign languages . 6 

3) 1 wish I could speak another 

language perfectly. 6 

4) I want to read the literature 
of another culture in the 
original language rather than 

a translation. 6 

5) I wish I could read newspapers 

and magazines in another 
language. 6 

6) I would really like to learn 

a lot of foreign languages . 6 

7) If I planned to stay in another 

country, I would make an effort 
to learn the language although 
I could get along in my native 
language . g 



5 4 

5 4 

5 4 

5 4 

5 4 

5. 4 

5 4 



3 2 

3 2 

3 2 

3 2 

3 2 

3 2 

3 2 



1 0 
1 0 
1 0 

1 0 

1 0 
1 0 

1 0 



8) I enjoy meeting and listening 

to people who speak other 
languages . 6 

9) Studying a foreign language 

is an enjoyable experience. 6 



5 4 3 

5 4 3 



2 10 
2 10 



Part 2 . These questions have to do with the role you might play 
in yo;ir son's/daughter's studying Korean. Please circle the 
number corresponding how true the statement is in your case. 



Absolutely 
true in my 
case 
6 5 



4 3 



1) I really encourage my 
child to study Korean. 

2) I try to help me child 
with his/her Korean 
homework . 

3) I feel that my child 
should continue studying 
Korean all through 
school . 



6 5 4 3 



6 5 4 3 



6 5 4 3 



4) I think my child should 
devote more time to 

his/her Korean studies. 6543 



Not true 
in my case 
at all 
2 10 



2 10 



2 10 



2 1 0 



2 10 



5) I show considerable interest 
in anything to do with my 

child's Korean class. 6 5 4 3 2 1 0 

6) I encoiirage my child 

to practice his/her Korean 

as much as possible. 6 5 43 2 1 0 

7) I have stressed to my 
child the importance 
Korean will have when 

he/she finishes school. 6543210 



8) I feel that my child should 
learn as much Korean as 

possible. 6543210 

9) I urge my child to get help 
from his/her teacher if he/she 
is having problems with 

his/her Korean class. 6543210 
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Part 3. Please check: 



GENDER: Male. 



Female. 



son 



"My 



IS in a 



daughter 



high school Korean class, 
high school Chinese class, 
high school Russian class, 
high school Japanese class 
elementary- level Chinese- 
Icinguage program, 
elementary- level Japanese . 
immersion program. 



What is your ethnic background? 

Caucasieui Native American 

; Latino Filipino 

Japanese Russian 

Chinese Other (Please specify: 

Korean 

Mixed ancestry (Please specify: 



What languages do you speak? ^ 

What is your first language? 

What language do you consider to be 

your domineuit language? 

If more than one leuiguage is spoken in your home, please list 
them according to how often they are used : 

Most-often used language 

2nd most-often used language 

3rd most-often used language 



Are you currently learning/ studying a second/f oreign language in 
class? 

If "Yes, " which language, and for how long have you been 
studying it? 



Language : 



Period of study:. 



years 



months 



Did you study a foreign language anytime when you were a student 
in school? Y®® 

If "Yes," which language (s), and for how long did you 



Study it/them? 

TjanoiiaCTR : 


Period 


of 


study : 


years 


months 


T^anguage : 


Period 


of 


study : 


years 


months 


Language: 


Period 


of 


study: 


years 


months 
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best copy available 



Appendix 6 



Teacher/Administrator Questionnaire 
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Teacher/ Administrator Questionnaire 

1 . In thinking about your program over the last four years of this grant, use the 3 point scale 
to indicate how successful you believe the program has been in the following areas: 

Success of Program 

Very Good Weak 

student interest to enroll in program 

student enthusiasm in learning language 

parent support of the program 

availability of learning materials 

implementation of curriculum 

2. Does your program have any of the following components 

(Check as many as apply) 

Yes No 

library of books/printed materials 

language laboratory 

computers with language software . 

parent support group for fund raising 

other (specify) 

3. Would you describe your project as a model program? 

Yes No 

Please explain your answer: 



ERIC 



1 
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4. Describe those aspects of your program which have worked well for you. 



5. Teaching any foreign language is difficult, but are there special difficulties that you (or 
your program) have encountered that make teaching your target language even more 
difficult. Please specify what these difficulties are, if any: 



6. Have you had any of the following types of classroom aides? If yes, on average how 
many hours per week? 

Yes No hrs. per week 

Project-supported aide 

Student aide 

Parent volunteer 



Community volunteer 



7. Which of the following types of support has your program received from a language 
heritage community and/or foreign consulate? (Check all that apply) 

Yes No 



Financial assistance (e.g., grant) 

Fundraising assistance 

Donations of printed materials 

Donations of computers, computer software 

Mentorship/intemship with local businesses 

Opportunities for students to interact 
(e.g., festivals, contests) with native speakers 



8. Some programs have received more administrative support than others. Using the 3 point 
scale provided please specify the level of support the program has received from the 
following: 

Level of Support 
High Average Low 



Department head (if applicable) 
Principal 

Site council (if applicable) 

Central administration (school district) 
Board of Education 



9. How would you describe your approach to language teaching? 



ERIC 
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10 . 



How would you summarize the goals of the program at the start of the project? 



11. Have these goals been achieved? 
Yes No 



12. Have the goals changed over the past four years, and if so, how ? 



13. What will happen to the program at your school after the grant ends? 



Please complete the following: 

Name 

School 

Language 





4 
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